[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:25 dist-test-slave-tk9z krb5kdc[29958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:25 dist-test-slave-tk9z krb5kdc[29958](info): set up 2 sockets
Aug 08 17:45:25 dist-test-slave-tk9z krb5kdc[29958](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 17:45:26.904417 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:26 dist-test-slave-tk9z krb5kdc[29958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:27.030354 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40675
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:27.113762 29979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:27.113991 29979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:27.114012 29979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:27.115338 29979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:27.115389 29979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:27.115410 29979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:27.115423 29979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:27.115464 29979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:27.116992 29979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:27.117237 29979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:27.117501 29979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:27.122726 29989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:27.122763 29993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:27.122807 29979 server_base.cc:1034] running on GCE node
W20260808 17:45:27.122747 29987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:27.123178 29979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:27.123507 29979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:27.124678 29979 hybrid_clock.cc:648] HybridClock initialized: now 1786211127124672 us; error 82 us; skew 500 ppm
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:27.127210 29979 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:27.127614 29979 webserver.cc:460] Webserver started at http://127.29.62.254:37009/ using document root <none> and password file <none>
I20260808 17:45:27.127801 29979 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:27.127840 29979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:27.127933 29979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:27.129045 29979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "e4778f9b87d849129cddfc43e036f166"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "f783a6b3f6d5e0e8cb6494a6676a09f6"
server_key_iv: "882f5c073ac777764c5d20229a310c92"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.129370 29979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "e4778f9b87d849129cddfc43e036f166"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "f783a6b3f6d5e0e8cb6494a6676a09f6"
server_key_iv: "882f5c073ac777764c5d20229a310c92"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.130753 29979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:27.131510 30002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.131693 29979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:45:27.131783 29979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "e4778f9b87d849129cddfc43e036f166"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "f783a6b3f6d5e0e8cb6494a6676a09f6"
server_key_iv: "882f5c073ac777764c5d20229a310c92"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.131843 29979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:27.144711 29979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:27.146911 29979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:27.147030 29979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:27.151103 29979 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40675
I20260808 17:45:27.151165 30075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40675 every 8 connection(s)
I20260808 17:45:27.151508 29979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:27.152467 30079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:27.155107 30079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166: Bootstrap starting.
I20260808 17:45:27.155406 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 29979
I20260808 17:45:27.155501 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:27.155776 30079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:27.156083 30079 log.cc:824] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:27.156152 29947 external_mini_cluster.cc:1428] Setting key dda98c99dcffcac2e14ebe8c4d4023dc
I20260808 17:45:27.156992 30079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166: No bootstrap required, opened a new log
I20260808 17:45:27.158849 30079 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4778f9b87d849129cddfc43e036f166" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40675 } }
I20260808 17:45:27.158995 30079 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:27.159018 30079 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4778f9b87d849129cddfc43e036f166, State: Initialized, Role: FOLLOWER
I20260808 17:45:27.159152 30079 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4778f9b87d849129cddfc43e036f166" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40675 } }
I20260808 17:45:27.159230 30079 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:27.159253 30079 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:27.159322 30079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:27.160177 30079 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4778f9b87d849129cddfc43e036f166" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40675 } }
I20260808 17:45:27.160271 30079 leader_election.cc:302] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4778f9b87d849129cddfc43e036f166; no voters:
I20260808 17:45:27.160506 30079 leader_election.cc:288] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:27.160524 30089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:27.160635 30089 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [term 1 LEADER]: Becoming Leader. State: Replica: e4778f9b87d849129cddfc43e036f166, State: Running, Role: LEADER
I20260808 17:45:27.160720 30089 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4778f9b87d849129cddfc43e036f166" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40675 } }
I20260808 17:45:27.160775 30079 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:27.161055 30089 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4778f9b87d849129cddfc43e036f166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4778f9b87d849129cddfc43e036f166" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40675 } } }
I20260808 17:45:27.161211 30089 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:27.161413 30089 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4778f9b87d849129cddfc43e036f166. Latest consensus state: current_term: 1 leader_uuid: "e4778f9b87d849129cddfc43e036f166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4778f9b87d849129cddfc43e036f166" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40675 } } }
I20260808 17:45:27.161541 30089 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:27.161916 30108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:27.162640 30108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:27.164242 30108 catalog_manager.cc:1347] Generated new cluster ID: 3ff8a5213b944fbc83f5a765329f5f59
I20260808 17:45:27.164296 30108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:27.173780 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.157199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.29.62.254:40675)
0808 17:45:27.157380 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:27.157388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:27.157678 (+ 290us) server_negotiation.cc:407] Connection header received
0808 17:45:27.159666 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:27.159671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:27.159712 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:27.159835 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:27.161925 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.163399 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.164865 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.165483 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.165600 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.169727 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:27.169731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:27.169731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:27.169742 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:27.171743 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.172253 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.172256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.172256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.172370 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.172668 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.172669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.172669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.172960 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:27.173110 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:27.173403 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 17:45:27.173530 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260808 17:45:27.176736 30108 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:27.177242 30108 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:27.181900 30108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4778f9b87d849129cddfc43e036f166: Generated new TSK 0
I20260808 17:45:27.182075 30108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:27.211333 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40675
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:27.314726 30129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:27.314929 30129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:27.314953 30129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:27.316102 30129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:27.316146 30129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:27.316174 30129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:27.317657 30129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40675
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:27.317902 30129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:27.318164 30129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:27.324049 30137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:27.324034 30135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:27.324036 30134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:27.324470 30129 server_base.cc:1034] running on GCE node
I20260808 17:45:27.324702 30129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:27.324944 30129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:27.326104 30129 hybrid_clock.cc:648] HybridClock initialized: now 1786211127326068 us; error 53 us; skew 500 ppm
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:27.328711 30129 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:27.329172 30129 webserver.cc:460] Webserver started at http://127.29.62.193:37375/ using document root <none> and password file <none>
I20260808 17:45:27.329407 30129 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:27.329475 30129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:27.329617 30129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:27.330788 30129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "3c43939205e54b389c70703e30f25695"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "6ec4f64689c246cea1688d389d4b3636"
server_key_iv: "7dd68f04719b558c3783a1924719a327"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.331164 30129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "3c43939205e54b389c70703e30f25695"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "6ec4f64689c246cea1688d389d4b3636"
server_key_iv: "7dd68f04719b558c3783a1924719a327"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.332614 30129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:27.333338 30146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.333530 30129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:27.333634 30129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "3c43939205e54b389c70703e30f25695"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "6ec4f64689c246cea1688d389d4b3636"
server_key_iv: "7dd68f04719b558c3783a1924719a327"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.333719 30129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:27.342120 30129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:27.343920 30129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:27.344077 30129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:27.344298 30129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:27.344605 30129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:27.344660 30129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.344700 30129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:27.344729 30129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.351420 30129 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:38095
I20260808 17:45:27.351514 30312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:38095 every 8 connection(s)
I20260808 17:45:27.351836 30129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:27.357554 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30129
I20260808 17:45:27.357661 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:27.357821 29947 external_mini_cluster.cc:1428] Setting key 44eedc6ca3e86ce48b42a712b7611c1c
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:27.362679 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.352713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:34883 (local address 127.29.62.254:40675)
0808 17:45:27.352771 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:27.352773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:27.352998 (+ 225us) server_negotiation.cc:407] Connection header received
0808 17:45:27.354032 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:27.354033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:27.354050 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:27.354114 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:27.354834 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.355361 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.356361 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.356577 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.356657 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.359362 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:27.359363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:27.359364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:27.359370 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:27.361049 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.361569 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.361571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.361571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.361636 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.361909 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.361911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.361911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.362061 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:27.362137 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:27.362492 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 17:45:27.362586 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:45:27.362887 30318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.352691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40675 (local address 127.29.62.193:34883)
0808 17:45:27.352869 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:27.352904 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:27.353911 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:27.354191 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:27.354195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:27.354530 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:27.354731 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:27.354736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.355462 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.355463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:27.356255 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:27.356258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.356666 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.356667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:27.356790 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.357353 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:27.357366 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:27.359273 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:27.361148 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:27.361155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:27.361157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:27.361489 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:27.361713 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:27.361714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:27.361714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:27.361824 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:27.362142 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:27.362150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:27.362323 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 17:45:27.362460 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 17:45:27.362630 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260808 17:45:27.363615 30313 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40675
I20260808 17:45:27.363703 30313 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:27.363903 30313 heartbeater.cc:511] Master 127.29.62.254:40675 requested a full tablet report, sending...
I20260808 17:45:27.364531 30025 ts_manager.cc:194] Registered new tserver with Master: 3c43939205e54b389c70703e30f25695 (127.29.62.193:38095)
I20260808 17:45:27.365517 30025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:34883
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:27.384459 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40675
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:27.469048 30321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:27.469260 30321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:27.469285 30321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:27.470508 30321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:27.470548 30321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:27.470575 30321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:27.472074 30321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40675
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:27.472321 30321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:27.472561 30321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:27.477638 30328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:27.477698 30327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:27.477867 30321 server_base.cc:1034] running on GCE node
W20260808 17:45:27.477643 30330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:27.478116 30321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:27.478317 30321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:27.479552 30321 hybrid_clock.cc:648] HybridClock initialized: now 1786211127479538 us; error 29 us; skew 500 ppm
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:27.482055 30321 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:27.482600 30321 webserver.cc:460] Webserver started at http://127.29.62.194:38737/ using document root <none> and password file <none>
I20260808 17:45:27.482836 30321 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:27.482880 30321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:27.483049 30321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:27.484097 30321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "5d0dbf576e1a4d8eadfba24967a7d8b8"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "42816ad82e85ee4c167b6760f7ca6443"
server_key_iv: "6f98c2941d129bb769c6502eed702e63"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.484468 30321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "5d0dbf576e1a4d8eadfba24967a7d8b8"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "42816ad82e85ee4c167b6760f7ca6443"
server_key_iv: "6f98c2941d129bb769c6502eed702e63"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.485967 30321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:45:27.486835 30343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.487039 30321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:45:27.487142 30321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "5d0dbf576e1a4d8eadfba24967a7d8b8"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "42816ad82e85ee4c167b6760f7ca6443"
server_key_iv: "6f98c2941d129bb769c6502eed702e63"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.487210 30321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:27.502151 30321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:27.504089 30321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:27.504204 30321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:27.504381 30321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:27.504722 30321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:27.504781 30321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.504813 30321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:27.504843 30321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.511431 30321 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:37737
I20260808 17:45:27.511474 30516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:37737 every 8 connection(s)
I20260808 17:45:27.511814 30321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:27.520478 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30321
I20260808 17:45:27.520584 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:27.520776 29947 external_mini_cluster.cc:1428] Setting key 68ab40f204afc4663c514d4adde04e69
I20260808 17:45:27.521584 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.512700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:50203 (local address 127.29.62.254:40675)
0808 17:45:27.512775 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:27.512777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:27.512993 (+ 216us) server_negotiation.cc:407] Connection header received
0808 17:45:27.513570 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:27.513572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:27.513585 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:27.513646 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:27.514691 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.515200 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.516234 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.516438 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.516536 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.518778 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:27.518780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:27.518780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:27.518788 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:27.520108 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.520507 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.520508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.520508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.520580 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.520879 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.520880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.520881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.521020 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:27.521107 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:27.521389 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 17:45:27.521494 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:45:27.521770 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.512689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40675 (local address 127.29.62.194:50203)
0808 17:45:27.512864 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:27.512901 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:27.513452 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:27.513725 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:27.513729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:27.514338 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:27.514575 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:27.514580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.515303 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.515304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:27.516129 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:27.516131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.516549 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.516550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:27.516685 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.517239 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:27.517247 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:27.518677 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:27.520181 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:27.520187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:27.520189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:27.520415 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:27.520664 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:27.520664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:27.520665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:27.520783 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:27.521098 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:27.521106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:27.521252 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 17:45:27.521366 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 17:45:27.521525 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260808 17:45:27.523932 30517 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40675
I20260808 17:45:27.524071 30517 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:27.524286 30517 heartbeater.cc:511] Master 127.29.62.254:40675 requested a full tablet report, sending...
I20260808 17:45:27.524691 30025 ts_manager.cc:194] Registered new tserver with Master: 5d0dbf576e1a4d8eadfba24967a7d8b8 (127.29.62.194:37737)
I20260808 17:45:27.525071 30025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:50203
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:27.547370 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40675
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:27.630950 30523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:27.631177 30523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:27.631217 30523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:27.632367 30523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:27.632432 30523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:27.632480 30523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:27.633931 30523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40675
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:27.634205 30523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:27.634498 30523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:27.639433 30533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:27.639449 30532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:27.639467 30535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:27.639771 30523 server_base.cc:1034] running on GCE node
I20260808 17:45:27.639964 30523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:27.640147 30523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:27.641283 30523 hybrid_clock.cc:648] HybridClock initialized: now 1786211127641269 us; error 29 us; skew 500 ppm
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:27.643812 30523 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:27.644223 30523 webserver.cc:460] Webserver started at http://127.29.62.195:45433/ using document root <none> and password file <none>
I20260808 17:45:27.644440 30523 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:27.644485 30523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:27.644635 30523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:27.645656 30523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "38fbf2376b844be1840961e071454a3f"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "5708b7330567f5030504661022532500"
server_key_iv: "c37f3ee0a2bd486df29ce616bc870286"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.646042 30523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "38fbf2376b844be1840961e071454a3f"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "5708b7330567f5030504661022532500"
server_key_iv: "c37f3ee0a2bd486df29ce616bc870286"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.647495 30523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:27.648264 30546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.648433 30523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:27.648535 30523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "38fbf2376b844be1840961e071454a3f"
format_stamp: "Formatted at 2026-08-08 17:45:27 on dist-test-slave-tk9z"
server_key: "5708b7330567f5030504661022532500"
server_key_iv: "c37f3ee0a2bd486df29ce616bc870286"
server_key_version: "encryptionkey@0"
I20260808 17:45:27.648612 30523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:27.667740 30523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:27.669684 30523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:27.669853 30523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:27.670111 30523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:27.670538 30523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:27.670595 30523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.670640 30523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:27.670667 30523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:27.676862 30523 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:45489
I20260808 17:45:27.676918 30729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:45489 every 8 connection(s)
I20260808 17:45:27.677233 30523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:45:27.683041 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30523
I20260808 17:45:27.683151 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:27.683331 29947 external_mini_cluster.cc:1428] Setting key 7d229d192f4ddf292f2e4c3a08790f2a
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:27.687103 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.677968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:45485 (local address 127.29.62.254:40675)
0808 17:45:27.678034 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:27.678036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:27.678252 (+ 216us) server_negotiation.cc:407] Connection header received
0808 17:45:27.678846 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:27.678847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:27.678860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:27.678936 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:27.679895 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.680409 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.681395 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.681613 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.681692 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.684392 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:27.684393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:27.684394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:27.684403 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:27.685615 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.686061 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.686063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.686063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.686126 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.686430 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.686432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.686432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.686556 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:27.686633 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:27.686927 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 17:45:27.687014 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:45:27.687294 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.677963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40675 (local address 127.29.62.195:45485)
0808 17:45:27.678137 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:27.678173 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:27.678741 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:27.679013 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:27.679017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:27.679596 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:27.679793 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:27.679798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.680511 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.680512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:27.681302 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:27.681305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.681699 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.681701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:27.681835 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.682458 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:27.682467 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:27.684288 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:27.685713 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:27.685720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:27.685722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:27.685962 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:27.686197 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:27.686198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:27.686199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:27.686324 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:27.686639 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:27.686647 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:27.686783 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 17:45:27.686889 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 17:45:27.687052 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260808 17:45:27.687757 30730 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40675
I20260808 17:45:27.687870 30730 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:27.688055 30730 heartbeater.cc:511] Master 127.29.62.254:40675 requested a full tablet report, sending...
I20260808 17:45:27.688429 30025 ts_manager.cc:194] Registered new tserver with Master: 38fbf2376b844be1840961e071454a3f (127.29.62.195:45489)
I20260808 17:45:27.688802 30025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:45485
I20260808 17:45:27.695389 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:27.719164 30741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.711508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.29.62.193:38095)
0808 17:45:27.711712 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:27.711716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:27.711727 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:45:27.711755 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:27.711756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:27.711771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:27.711846 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:27.712387 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.713112 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.713823 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.714160 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.714222 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.716463 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:27.716471 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:27.716471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:27.716478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:27.717920 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.718241 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.718243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.718243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.718302 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.718525 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.718526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.718526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.718665 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:27.718793 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:27.718931 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 17:45:27.719026 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":117,"threads_started":1}
W20260808 17:45:27.719573 30186 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43318
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[29958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:27.742379 30741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:27.735871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.29.62.193:38095)
0808 17:45:27.735936 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:27.735938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:27.736054 (+ 116us) server_negotiation.cc:407] Connection header received
0808 17:45:27.736102 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:27.736103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:27.736114 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:27.736161 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:27.736669 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.737158 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.737840 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:27.738046 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:27.738099 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:27.740290 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:27.740291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:27.740292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:27.740299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:27.741465 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.741760 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.741761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.741761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.741811 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:27.741986 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:27.741986 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:27.741987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:27.742077 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:27.742164 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:27.742229 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:45:27.742286 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:45:27.742971 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30129
I20260808 17:45:27.769244 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30321
I20260808 17:45:27.777953 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30523
I20260808 17:45:27.786079 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 29979
2026-08-08T17:45:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2164 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[30752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[30752](info): set up 2 sockets
Aug 08 17:45:27 dist-test-slave-tk9z krb5kdc[30752](info): commencing operation
krb5kdc: starting...
W20260808 17:45:29.036751 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:29.157784 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:36251
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:36251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:29.241370 30773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:29.241633 30773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:29.241671 30773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:29.242923 30773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:29.242993 30773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:29.243018 30773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:29.243038 30773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:29.243069 30773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:29.244547 30773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:36251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:36251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:29.244848 30773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:29.245080 30773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:29.249893 30782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:29.249938 30785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:29.250137 30773 server_base.cc:1034] running on GCE node
W20260808 17:45:29.249950 30783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:29.250501 30773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:29.250805 30773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:29.251973 30773 hybrid_clock.cc:648] HybridClock initialized: now 1786211129251955 us; error 31 us; skew 500 ppm
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:29.254565 30773 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:29.255086 30773 webserver.cc:460] Webserver started at http://127.29.62.254:42193/ using document root <none> and password file <none>
I20260808 17:45:29.255321 30773 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:29.255388 30773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:29.255532 30773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:29.256613 30773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "dc81267caeea483991b355bb01df31ca"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "cde2d6f78e6addd7529ddd4ab197eeea"
server_key_iv: "5e1f6196d9fbbdc7b5b540cc92e8fda2"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.256995 30773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "dc81267caeea483991b355bb01df31ca"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "cde2d6f78e6addd7529ddd4ab197eeea"
server_key_iv: "5e1f6196d9fbbdc7b5b540cc92e8fda2"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.258342 30773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:45:29.259143 30801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.259318 30773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:29.259438 30773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "dc81267caeea483991b355bb01df31ca"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "cde2d6f78e6addd7529ddd4ab197eeea"
server_key_iv: "5e1f6196d9fbbdc7b5b540cc92e8fda2"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.259523 30773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:29.272670 30773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:29.274696 30773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:29.274852 30773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:29.278795 30773 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:36251
I20260808 17:45:29.278829 30876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:36251 every 8 connection(s)
I20260808 17:45:29.279196 30773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:29.280102 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:29.282469 30877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca: Bootstrap starting.
I20260808 17:45:29.283088 30877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:29.283382 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30773
I20260808 17:45:29.283392 30877 log.cc:824] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:29.283499 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:29.283658 29947 external_mini_cluster.cc:1428] Setting key e7c8fcdda440f7fd78b7f7609bbdc4c0
I20260808 17:45:29.284336 30877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca: No bootstrap required, opened a new log
I20260808 17:45:29.286655 30877 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc81267caeea483991b355bb01df31ca" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36251 } }
I20260808 17:45:29.286827 30877 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:29.286880 30877 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc81267caeea483991b355bb01df31ca, State: Initialized, Role: FOLLOWER
I20260808 17:45:29.287019 30877 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc81267caeea483991b355bb01df31ca" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36251 } }
I20260808 17:45:29.287104 30877 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:29.287148 30877 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:29.287196 30877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:29.287757 30877 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc81267caeea483991b355bb01df31ca" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36251 } }
I20260808 17:45:29.287878 30877 leader_election.cc:302] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc81267caeea483991b355bb01df31ca; no voters:
I20260808 17:45:29.288122 30877 leader_election.cc:288] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:29.288148 30884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:29.288360 30884 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [term 1 LEADER]: Becoming Leader. State: Replica: dc81267caeea483991b355bb01df31ca, State: Running, Role: LEADER
I20260808 17:45:29.288414 30877 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:29.288563 30884 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc81267caeea483991b355bb01df31ca" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36251 } }
I20260808 17:45:29.288975 30886 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc81267caeea483991b355bb01df31ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc81267caeea483991b355bb01df31ca" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36251 } } }
I20260808 17:45:29.289145 30886 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:29.289705 30899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:29.290146 30885 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc81267caeea483991b355bb01df31ca. Latest consensus state: current_term: 1 leader_uuid: "dc81267caeea483991b355bb01df31ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc81267caeea483991b355bb01df31ca" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36251 } } }
I20260808 17:45:29.290236 30899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:29.290230 30885 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:29.292316 30899 catalog_manager.cc:1347] Generated new cluster ID: a66f4f11bea24e0b9e9dbd6c256abe70
I20260808 17:45:29.292366 30899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:29.295035 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.284218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47188 (local address 127.29.62.254:36251)
0808 17:45:29.284409 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.284420 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.284463 (+ 43us) server_negotiation.cc:407] Connection header received
0808 17:45:29.284904 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.284910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.284956 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.285107 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:29.285740 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.286690 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.287622 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.288125 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.288246 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.291196 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:29.291200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:29.291200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:29.291212 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:29.293325 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.293748 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.293751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.293751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.293852 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.294072 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.294073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.294073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.294349 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:29.294541 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.294661 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.294791 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260808 17:45:29.311993 30899 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:29.312743 30899 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:29.318219 30899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc81267caeea483991b355bb01df31ca: Generated new TSK 0
I20260808 17:45:29.318460 30899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:29.343426 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:36251
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:29.425884 30918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:29.426128 30918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:29.426168 30918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:29.427420 30918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:29.427500 30918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:29.427549 30918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:29.429081 30918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:36251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:29.429363 30918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:29.429605 30918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:29.434629 30927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:29.434711 30925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:29.434824 30918 server_base.cc:1034] running on GCE node
W20260808 17:45:29.434629 30924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:29.435065 30918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:29.435313 30918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:29.436473 30918 hybrid_clock.cc:648] HybridClock initialized: now 1786211129436455 us; error 38 us; skew 500 ppm
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:29.439083 30918 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:29.439558 30918 webserver.cc:460] Webserver started at http://127.29.62.193:37367/ using document root <none> and password file <none>
I20260808 17:45:29.439816 30918 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:29.439886 30918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:29.440037 30918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:29.441111 30918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "a8b586e723eb409bab8ffde60665af55"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "8ad852845c3c41ba5d0a4a6224f570ba"
server_key_iv: "705bd52dbf9544578f3004efe81100a6"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.441491 30918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "a8b586e723eb409bab8ffde60665af55"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "8ad852845c3c41ba5d0a4a6224f570ba"
server_key_iv: "705bd52dbf9544578f3004efe81100a6"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.442999 30918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:29.443745 30942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.443917 30918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:29.444024 30918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "a8b586e723eb409bab8ffde60665af55"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "8ad852845c3c41ba5d0a4a6224f570ba"
server_key_iv: "705bd52dbf9544578f3004efe81100a6"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.444098 30918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:29.465798 30918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:29.467780 30918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:29.467962 30918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:29.468212 30918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:29.468545 30918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:29.468602 30918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.468647 30918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:29.468690 30918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.474599 30918 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:42673
I20260808 17:45:29.474653 31123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:42673 every 8 connection(s)
I20260808 17:45:29.474965 30918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:29.479071 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30918
I20260808 17:45:29.479178 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:29.479337 29947 external_mini_cluster.cc:1428] Setting key a0f278ae76166b90772060480edf5a90
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:29.484946 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.475663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:43571 (local address 127.29.62.254:36251)
0808 17:45:29.475726 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.475728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.475974 (+ 246us) server_negotiation.cc:407] Connection header received
0808 17:45:29.476973 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.476974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.476987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.477037 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:29.477668 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.478173 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.479217 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.479525 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.479603 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.481981 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:29.481983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:29.481983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:29.481990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:29.483344 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.483849 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.483850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.483850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.483904 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.484225 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.484225 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.484226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.484384 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:29.484459 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.484806 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.484876 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:45:29.485244 31126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.475623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:36251 (local address 127.29.62.193:43571)
0808 17:45:29.475823 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:29.475876 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:29.476857 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:29.477107 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:29.477111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:29.477375 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:29.477564 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.477568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.478273 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.478275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.479106 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.479109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.479637 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.479638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.479792 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.480358 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:29.480366 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:29.481870 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:29.483430 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:29.483437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:29.483439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:29.483767 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:29.483984 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:29.483985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:29.483986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:29.484145 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:29.484476 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:29.484484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:29.484660 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 17:45:29.484786 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 17:45:29.484965 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260808 17:45:29.485848 31124 heartbeater.cc:347] Connected to a master server at 127.29.62.254:36251
I20260808 17:45:29.485960 31124 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:29.486153 31124 heartbeater.cc:511] Master 127.29.62.254:36251 requested a full tablet report, sending...
I20260808 17:45:29.486719 30824 ts_manager.cc:194] Registered new tserver with Master: a8b586e723eb409bab8ffde60665af55 (127.29.62.193:42673)
I20260808 17:45:29.487318 30824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:43571
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:29.504889 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:36251
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:29.587240 31130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:29.587451 31130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:29.587491 31130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:29.588613 31130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:29.588652 31130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:29.588680 31130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:29.590420 31130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:36251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:29.590608 31130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:29.590772 31130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:29.595512 31142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:29.595504 31150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:29.595505 31154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:29.595759 31130 server_base.cc:1034] running on GCE node
I20260808 17:45:29.595986 31130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:29.596218 31130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:29.597390 31130 hybrid_clock.cc:648] HybridClock initialized: now 1786211129597368 us; error 37 us; skew 500 ppm
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:29.599917 31130 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:29.600332 31130 webserver.cc:460] Webserver started at http://127.29.62.194:36207/ using document root <none> and password file <none>
I20260808 17:45:29.600564 31130 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:29.600631 31130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:29.600782 31130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:29.601775 31130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "f555c5231faf45388d664e1d6f06c991"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "271cbbb2e0aeb348a41be6fe9da685d9"
server_key_iv: "5d751799b20a606bda8254529a1f4230"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.602135 31130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "f555c5231faf45388d664e1d6f06c991"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "271cbbb2e0aeb348a41be6fe9da685d9"
server_key_iv: "5d751799b20a606bda8254529a1f4230"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.603530 31130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:29.604223 31162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.604406 31130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:29.604508 31130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "f555c5231faf45388d664e1d6f06c991"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "271cbbb2e0aeb348a41be6fe9da685d9"
server_key_iv: "5d751799b20a606bda8254529a1f4230"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.604592 31130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:29.623618 31130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:29.625425 31130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:29.625588 31130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:29.625813 31130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:29.626137 31130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:29.626194 31130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.626236 31130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:29.626266 31130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.632239 31130 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:41597
I20260808 17:45:29.632280 31328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:41597 every 8 connection(s)
I20260808 17:45:29.632633 31130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:29.640082 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31130
I20260808 17:45:29.640188 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:29.640383 29947 external_mini_cluster.cc:1428] Setting key 0d369198ca8499628e31ccd4b78caff3
I20260808 17:45:29.642416 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.633397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:44945 (local address 127.29.62.254:36251)
0808 17:45:29.633481 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.633483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.633656 (+ 173us) server_negotiation.cc:407] Connection header received
0808 17:45:29.634723 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.634725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.634738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.634789 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:29.635421 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.635921 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.636994 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.637193 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.637269 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.639450 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:29.639451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:29.639451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:29.639458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:29.640977 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.641390 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.641392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.641392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.641458 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.641726 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.641726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.641727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.641841 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:29.641918 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.642193 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.642279 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:45:29.642643 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.633340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:36251 (local address 127.29.62.194:44945)
0808 17:45:29.633531 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:29.633577 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:29.634603 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:29.634859 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:29.634863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:29.635130 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:29.635313 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.635317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.636019 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.636021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.636890 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.636892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.637275 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.637277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.637407 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.637960 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:29.637968 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:29.639357 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:29.641066 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:29.641074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:29.641076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:29.641308 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:29.641551 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:29.641552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:29.641552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:29.641649 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:29.641908 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:29.641915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:29.642054 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 17:45:29.642170 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 17:45:29.642329 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260808 17:45:29.643121 31329 heartbeater.cc:347] Connected to a master server at 127.29.62.254:36251
I20260808 17:45:29.643215 31329 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:29.643409 31329 heartbeater.cc:511] Master 127.29.62.254:36251 requested a full tablet report, sending...
I20260808 17:45:29.643774 30824 ts_manager.cc:194] Registered new tserver with Master: f555c5231faf45388d664e1d6f06c991 (127.29.62.194:41597)
I20260808 17:45:29.644160 30824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:44945
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:29.666723 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:36251
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:29.750391 31337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:29.750630 31337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:29.750669 31337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:29.752020 31337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:29.752089 31337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:29.752140 31337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:29.753685 31337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:36251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:29.753965 31337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:29.754204 31337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:29.759065 31343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:29.759070 31347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:29.759244 31342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:29.759274 31337 server_base.cc:1034] running on GCE node
I20260808 17:45:29.759624 31337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:29.759894 31337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:29.761056 31337 hybrid_clock.cc:648] HybridClock initialized: now 1786211129761046 us; error 39 us; skew 500 ppm
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:29.763682 31337 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:29.764063 31337 webserver.cc:460] Webserver started at http://127.29.62.195:45415/ using document root <none> and password file <none>
I20260808 17:45:29.764261 31337 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:29.764300 31337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:29.764389 31337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:29.765363 31337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "13efdba13acd42ca942a63d2ea0b841f"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "5f5afc88474f8016ed574ef605a6acd0"
server_key_iv: "acf0b316304cb75c5b57ad0869ab5ee9"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.765677 31337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "13efdba13acd42ca942a63d2ea0b841f"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "5f5afc88474f8016ed574ef605a6acd0"
server_key_iv: "acf0b316304cb75c5b57ad0869ab5ee9"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.767078 31337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:29.767776 31359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.767967 31337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:29.768051 31337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "13efdba13acd42ca942a63d2ea0b841f"
format_stamp: "Formatted at 2026-08-08 17:45:29 on dist-test-slave-tk9z"
server_key: "5f5afc88474f8016ed574ef605a6acd0"
server_key_iv: "acf0b316304cb75c5b57ad0869ab5ee9"
server_key_version: "encryptionkey@0"
I20260808 17:45:29.768107 31337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:29.775727 31337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:29.777446 31337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:29.777601 31337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:29.777812 31337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:29.778115 31337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:29.778147 31337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.778168 31337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:29.778213 31337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:29.784795 31337 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:44205
I20260808 17:45:29.784873 31535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:44205 every 8 connection(s)
I20260808 17:45:29.785174 31337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:29.792044 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31337
I20260808 17:45:29.792155 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:29.792320 29947 external_mini_cluster.cc:1428] Setting key 7570d6a26d65aa3cc77d64dc2f8c86fa
I20260808 17:45:29.794744 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.785891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:48123 (local address 127.29.62.254:36251)
0808 17:45:29.785936 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.785938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.786120 (+ 182us) server_negotiation.cc:407] Connection header received
0808 17:45:29.787115 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.787116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.787138 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.787190 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:29.787780 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.788308 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.789412 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.789697 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.789786 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.791896 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:29.791897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:29.791898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:29.791904 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:29.793516 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.793897 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.793898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.793898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.793950 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.794168 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.794168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.794169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.794269 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:29.794321 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.794616 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.794679 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:45:29.794911 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.785852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:36251 (local address 127.29.62.195:48123)
0808 17:45:29.786005 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:29.786037 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:29.787020 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:29.787242 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:29.787244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:29.787480 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:29.787679 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.787683 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.788414 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.788415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.789317 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.789319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.789792 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.789793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.789893 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.790470 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:29.790479 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:29.791785 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:29.793608 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:29.793614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:29.793615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:29.793827 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:29.794009 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:29.794010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:29.794010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:29.794111 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:29.794325 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:29.794329 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:29.794514 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 17:45:29.794599 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:45:29.794714 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260808 17:45:29.795347 31537 heartbeater.cc:347] Connected to a master server at 127.29.62.254:36251
I20260808 17:45:29.795433 31537 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:29.795571 31537 heartbeater.cc:511] Master 127.29.62.254:36251 requested a full tablet report, sending...
I20260808 17:45:29.795881 30824 ts_manager.cc:194] Registered new tserver with Master: 13efdba13acd42ca942a63d2ea0b841f (127.29.62.195:44205)
I20260808 17:45:29.796247 30824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:48123
I20260808 17:45:29.804101 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:45:29.812057 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.805768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47204 (local address 127.29.62.254:36251)
0808 17:45:29.805846 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.805848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.805969 (+ 121us) server_negotiation.cc:407] Connection header received
0808 17:45:29.806050 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.806051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.806067 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.806129 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:29.806763 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.807295 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.808016 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.808206 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.808273 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.809481 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:29.809482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:29.809482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:29.809490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:29.810884 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.811242 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.811243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.811243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.811302 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.811503 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.811504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.811504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.811658 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:29.811766 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.811872 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.811955 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 17:45:29.814143 30824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:29.815897 30824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:29.822403 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.818137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:41597 (local address 127.0.0.1:35964)
0808 17:45:29.818294 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:29.818332 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:29.818429 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:29.819065 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:29.819066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:29.819068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:29.819116 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.819117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.820367 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.820368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.821403 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.821405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.822047 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.822048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.822144 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.822160 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:45:29.822235 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 17:45:29.822301 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":53,"threads_started":1}
I20260808 17:45:29.822501 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.818447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.29.62.195:44205)
0808 17:45:29.818623 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.818627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.818639 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:45:29.818698 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.818700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.818715 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.818788 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:29.819232 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.819932 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.821371 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.822101 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.822175 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.822225 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.822346 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.822428 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260808 17:45:29.822475 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.818165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:44205 (local address 127.0.0.1:60380)
0808 17:45:29.818515 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:29.818524 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:29.818560 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:29.818894 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:29.818895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:29.818897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:29.818952 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.818954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.820037 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.820038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.821273 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.821276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.822173 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.822174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.822265 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.822268 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:29.822321 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 17:45:29.822355 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":1,"thread_start_us":56,"threads_started":1}
I20260808 17:45:29.822806 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.818614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.29.62.194:41597)
0808 17:45:29.818858 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.818862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.818873 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:45:29.818909 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.818911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.818926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.819070 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:29.819179 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.820243 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.821482 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.821955 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.822045 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.822105 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.822649 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.822703 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":97,"threads_started":1}
I20260808 17:45:29.823647 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.818137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42673 (local address 127.0.0.1:36194)
0808 17:45:29.818295 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:29.818342 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:29.818452 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:29.818890 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:29.818894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:29.818896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:29.818974 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.818976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.821482 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.821482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.822557 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.822560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.823390 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.823392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.823455 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.823459 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:29.823541 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:45:29.823581 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260808 17:45:29.823571 31430 tablet_service.cc:1467] Processing CreateTablet for tablet fd9adfbcddbd48f3b3885b30e4bb7a1e (DEFAULT_TABLE table=test-table [id=c0116fd7784744d3868165c11e9de21e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:29.823522 31233 tablet_service.cc:1467] Processing CreateTablet for tablet fd9adfbcddbd48f3b3885b30e4bb7a1e (DEFAULT_TABLE table=test-table [id=c0116fd7784744d3868165c11e9de21e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:29.823671 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.818355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36194 (local address 127.29.62.193:42673)
0808 17:45:29.818589 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.818595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.818608 (+ 13us) server_negotiation.cc:407] Connection header received
0808 17:45:29.818652 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.818654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.818670 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.818761 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:29.819077 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.820229 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.822654 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.823287 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.823375 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.823437 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.823559 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.823599 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":111,"threads_started":1}
I20260808 17:45:29.823899 31233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd9adfbcddbd48f3b3885b30e4bb7a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:29.823899 31430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd9adfbcddbd48f3b3885b30e4bb7a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:29.824560 31025 tablet_service.cc:1467] Processing CreateTablet for tablet fd9adfbcddbd48f3b3885b30e4bb7a1e (DEFAULT_TABLE table=test-table [id=c0116fd7784744d3868165c11e9de21e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:29.824872 31025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd9adfbcddbd48f3b3885b30e4bb7a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:29.827407 31560 tablet_bootstrap.cc:492] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991: Bootstrap starting.
I20260808 17:45:29.827632 31562 tablet_bootstrap.cc:492] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55: Bootstrap starting.
I20260808 17:45:29.827631 31561 tablet_bootstrap.cc:492] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f: Bootstrap starting.
I20260808 17:45:29.828378 31562 tablet_bootstrap.cc:654] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:29.828378 31561 tablet_bootstrap.cc:654] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:29.829020 31561 log.cc:824] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:29.829020 31562 log.cc:824] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:29.829859 31560 tablet_bootstrap.cc:654] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:29.829936 31562 tablet_bootstrap.cc:492] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55: No bootstrap required, opened a new log
I20260808 17:45:29.829944 31561 tablet_bootstrap.cc:492] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f: No bootstrap required, opened a new log
I20260808 17:45:29.830015 31562 ts_tablet_manager.cc:1397] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:45:29.830014 31561 ts_tablet_manager.cc:1397] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:45:29.830494 31560 log.cc:824] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:29.831175 31560 tablet_bootstrap.cc:492] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991: No bootstrap required, opened a new log
I20260808 17:45:29.831255 31560 ts_tablet_manager.cc:1397] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 17:45:29.832319 31561 raft_consensus.cc:348] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.832319 31562 raft_consensus.cc:348] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.832523 31561 raft_consensus.cc:374] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:29.832523 31562 raft_consensus.cc:374] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:29.832566 31561 raft_consensus.cc:729] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13efdba13acd42ca942a63d2ea0b841f, State: Initialized, Role: FOLLOWER
I20260808 17:45:29.832566 31562 raft_consensus.cc:729] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8b586e723eb409bab8ffde60665af55, State: Initialized, Role: FOLLOWER
I20260808 17:45:29.832703 31562 consensus_queue.cc:260] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.832764 31561 consensus_queue.cc:260] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.832791 31560 raft_consensus.cc:348] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.832988 31560 raft_consensus.cc:374] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:29.833024 31562 ts_tablet_manager.cc:1428] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:45:29.833034 31560 raft_consensus.cc:729] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f555c5231faf45388d664e1d6f06c991, State: Initialized, Role: FOLLOWER
I20260808 17:45:29.833045 31561 ts_tablet_manager.cc:1428] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:45:29.833160 31124 heartbeater.cc:503] Master 127.29.62.254:36251 was elected leader, sending a full tablet report...
I20260808 17:45:29.833160 31537 heartbeater.cc:503] Master 127.29.62.254:36251 was elected leader, sending a full tablet report...
I20260808 17:45:29.833168 31560 consensus_queue.cc:260] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.833492 31329 heartbeater.cc:503] Master 127.29.62.254:36251 was elected leader, sending a full tablet report...
I20260808 17:45:29.833478 31560 ts_tablet_manager.cc:1428] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:45:29.838991 31568 raft_consensus.cc:482] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:29.839062 31568 raft_consensus.cc:504] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.839346 31568 leader_election.cc:288] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f555c5231faf45388d664e1d6f06c991 (127.29.62.194:41597), 13efdba13acd42ca942a63d2ea0b841f (127.29.62.195:44205)
I20260808 17:45:29.842187 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.839544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:57845 (local address 127.29.62.194:41597)
0808 17:45:29.839612 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.839614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.839645 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:45:29.839738 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.839740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.839749 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.839834 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:29.839996 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.840495 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.841463 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.841844 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.841935 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.841944 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.842081 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.842124 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:45:29.842228 31126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.839466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:41597 (local address 127.29.62.193:57845)
0808 17:45:29.839590 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:29.839603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:29.839663 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:29.839838 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:29.839839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:29.839840 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:29.839926 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.839928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.840612 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.840613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.841364 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.841367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.841928 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.841929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.842004 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.842007 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:29.842057 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:45:29.842112 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":35}
I20260808 17:45:29.842252 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.839480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:44205 (local address 127.29.62.193:42309)
0808 17:45:29.839635 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:29.839645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:29.839710 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:29.839839 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:29.839839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:29.839841 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:29.839926 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.839927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.840560 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.840561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.841386 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:29.841389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.841953 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.841954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:29.842035 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.842038 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:29.842111 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 17:45:29.842150 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":47,"thread_start_us":91,"threads_started":1}
I20260808 17:45:29.842564 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.839552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:42309 (local address 127.29.62.195:44205)
0808 17:45:29.839594 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.839597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.839713 (+ 116us) server_negotiation.cc:407] Connection header received
0808 17:45:29.839759 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.839760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.839768 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.839822 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:29.840017 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.840461 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.841504 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.841848 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.841947 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.841954 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.842435 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.842483 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:45:29.842562 31259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd9adfbcddbd48f3b3885b30e4bb7a1e" candidate_uuid: "a8b586e723eb409bab8ffde60665af55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f555c5231faf45388d664e1d6f06c991" is_pre_election: true
I20260808 17:45:29.842701 31259 raft_consensus.cc:2454] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8b586e723eb409bab8ffde60665af55 in term 0.
I20260808 17:45:29.842891 31463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd9adfbcddbd48f3b3885b30e4bb7a1e" candidate_uuid: "a8b586e723eb409bab8ffde60665af55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13efdba13acd42ca942a63d2ea0b841f" is_pre_election: true
I20260808 17:45:29.842937 30944 leader_election.cc:302] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8b586e723eb409bab8ffde60665af55, f555c5231faf45388d664e1d6f06c991; no voters:
I20260808 17:45:29.843039 31463 raft_consensus.cc:2454] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8b586e723eb409bab8ffde60665af55 in term 0.
I20260808 17:45:29.843065 31568 raft_consensus.cc:2781] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:29.843107 31568 raft_consensus.cc:482] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:29.843142 31568 raft_consensus.cc:3037] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:29.843799 31568 raft_consensus.cc:504] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.843948 31568 leader_election.cc:288] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [CANDIDATE]: Term 1 election: Requested vote from peers f555c5231faf45388d664e1d6f06c991 (127.29.62.194:41597), 13efdba13acd42ca942a63d2ea0b841f (127.29.62.195:44205)
I20260808 17:45:29.844116 31463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd9adfbcddbd48f3b3885b30e4bb7a1e" candidate_uuid: "a8b586e723eb409bab8ffde60665af55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13efdba13acd42ca942a63d2ea0b841f"
I20260808 17:45:29.844108 31259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd9adfbcddbd48f3b3885b30e4bb7a1e" candidate_uuid: "a8b586e723eb409bab8ffde60665af55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f555c5231faf45388d664e1d6f06c991"
I20260808 17:45:29.844195 31463 raft_consensus.cc:3037] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:29.844195 31259 raft_consensus.cc:3037] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:29.845144 31463 raft_consensus.cc:2454] T fd9adfbcddbd48f3b3885b30e4bb7a1e P 13efdba13acd42ca942a63d2ea0b841f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8b586e723eb409bab8ffde60665af55 in term 1.
I20260808 17:45:29.845193 31259 raft_consensus.cc:2454] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8b586e723eb409bab8ffde60665af55 in term 1.
I20260808 17:45:29.845335 30948 leader_election.cc:302] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13efdba13acd42ca942a63d2ea0b841f, a8b586e723eb409bab8ffde60665af55; no voters:
I20260808 17:45:29.845427 31568 raft_consensus.cc:2781] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:29.845597 31568 raft_consensus.cc:686] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [term 1 LEADER]: Becoming Leader. State: Replica: a8b586e723eb409bab8ffde60665af55, State: Running, Role: LEADER
I20260808 17:45:29.845695 31568 consensus_queue.cc:237] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } }
I20260808 17:45:29.846498 30823 catalog_manager.cc:5495] T fd9adfbcddbd48f3b3885b30e4bb7a1e P a8b586e723eb409bab8ffde60665af55 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8b586e723eb409bab8ffde60665af55 (127.29.62.193). New cstate: current_term: 1 leader_uuid: "a8b586e723eb409bab8ffde60665af55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f555c5231faf45388d664e1d6f06c991" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8b586e723eb409bab8ffde60665af55" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13efdba13acd42ca942a63d2ea0b841f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 44205 } health_report { overall_health: UNKNOWN } } }
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:29.856875 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.849355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36210 (local address 127.29.62.193:42673)
0808 17:45:29.849424 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.849426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.849538 (+ 112us) server_negotiation.cc:407] Connection header received
0808 17:45:29.849614 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.849615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.849625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.849681 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:29.850279 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.850802 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.851643 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.851845 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.851918 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.854235 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:29.854243 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:29.854243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:29.854250 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:29.855765 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.856091 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.856094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.856094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.856151 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.856341 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.856342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.856343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.856486 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:29.856630 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.856687 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.856752 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:29.881448 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.874160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.29.62.193:42673)
0808 17:45:29.874231 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.874233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.874319 (+ 86us) server_negotiation.cc:407] Connection header received
0808 17:45:29.874401 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.874402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.874414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.874480 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:29.875052 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.875608 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.876320 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.876520 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.876591 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.879036 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:29.879037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:29.879038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:29.879044 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:29.880308 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.880646 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.880647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.880648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.880706 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.880882 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.880883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.880883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.880982 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:29.881068 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.881202 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.881326 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 17:45:29.883242 31330 tablet.cc:2369] T fd9adfbcddbd48f3b3885b30e4bb7a1e P f555c5231faf45388d664e1d6f06c991: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:45:29 dist-test-slave-tk9z krb5kdc[30752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:29.905014 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:29.898448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36224 (local address 127.29.62.193:42673)
0808 17:45:29.898508 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:29.898510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:29.898606 (+ 96us) server_negotiation.cc:407] Connection header received
0808 17:45:29.898652 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:29.898653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:29.898662 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:29.898713 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:29.899232 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.899698 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.900360 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:29.900540 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:29.900587 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:29.902756 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:29.902758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:29.902758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:29.902765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:29.903985 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.904319 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.904320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.904321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.904374 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:29.904578 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:29.904579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:29.904579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:29.904675 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:29.904762 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:29.904874 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 17:45:29.904946 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:45:29.905596 30991 tablet_service.cc:2908] Rejecting scan request for tablet fd9adfbcddbd48f3b3885b30e4bb7a1e: Uninitialized: clean time has not yet been initialized
I20260808 17:45:29.906591 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30918
I20260808 17:45:29.914799 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31130
I20260808 17:45:29.928709 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31337
I20260808 17:45:29.948657 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30773
2026-08-08T17:45:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2167 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:30 dist-test-slave-tk9z krb5kdc[31602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:30 dist-test-slave-tk9z krb5kdc[31602](info): set up 2 sockets
Aug 08 17:45:30 dist-test-slave-tk9z krb5kdc[31602](info): commencing operation
krb5kdc: starting...
W20260808 17:45:31.208921 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:31.332512 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:37141
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:37141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:31.414863 31622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:31.415132 31622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:31.415172 31622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:31.416340 31622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:31.416414 31622 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 17:45:31.416435 31622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:31.416460 31622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:31.416489 31622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:31.416512 31622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:31.418067 31622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:37141
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:37141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:31.418345 31622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:31.418615 31622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:31.423506 31628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:31.423563 31630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:31.423801 31622 server_base.cc:1034] running on GCE node
W20260808 17:45:31.423511 31632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:31.424144 31622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:31.424466 31622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:31.425638 31622 hybrid_clock.cc:648] HybridClock initialized: now 1786211131425626 us; error 44 us; skew 500 ppm
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:31.433060 31622 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:31.433527 31622 webserver.cc:460] Webserver started at http://127.29.62.254:37429/ using document root <none> and password file <none>
I20260808 17:45:31.433784 31622 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:31.433854 31622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:31.434010 31622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:31.435158 31622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "f3ace0bf5567410ea16c36fb02155510"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "e0e73f8819f87316bd053eed6216f273"
server_key_iv: "ea092f157559e7e2bc0a707bebbc2f84"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.435540 31622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "f3ace0bf5567410ea16c36fb02155510"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "e0e73f8819f87316bd053eed6216f273"
server_key_iv: "ea092f157559e7e2bc0a707bebbc2f84"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.436877 31622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:31.437673 31644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:31.437848 31622 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:31.437958 31622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "f3ace0bf5567410ea16c36fb02155510"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "e0e73f8819f87316bd053eed6216f273"
server_key_iv: "ea092f157559e7e2bc0a707bebbc2f84"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.438035 31622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:31.454658 31622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:31.456947 31622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:31.457131 31622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:31.462195 31622 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:37141
I20260808 17:45:31.462309 31735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:37141 every 8 connection(s)
I20260808 17:45:31.462656 31622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:31.463335 31736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:31.465629 31736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510: Bootstrap starting.
I20260808 17:45:31.466179 31736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:31.466436 31736 log.cc:824] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:31.467123 31736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510: No bootstrap required, opened a new log
I20260808 17:45:31.467754 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31622
I20260808 17:45:31.467862 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:31.468058 29947 external_mini_cluster.cc:1428] Setting key cacd15a233d2593c972f14c7483cd859
I20260808 17:45:31.469084 31736 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ace0bf5567410ea16c36fb02155510" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37141 } }
I20260808 17:45:31.469259 31736 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:31.469301 31736 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ace0bf5567410ea16c36fb02155510, State: Initialized, Role: FOLLOWER
I20260808 17:45:31.469442 31736 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ace0bf5567410ea16c36fb02155510" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37141 } }
I20260808 17:45:31.469537 31736 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:31.469579 31736 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:31.469633 31736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:31.470329 31736 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ace0bf5567410ea16c36fb02155510" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37141 } }
I20260808 17:45:31.470489 31736 leader_election.cc:302] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3ace0bf5567410ea16c36fb02155510; no voters:
I20260808 17:45:31.470700 31736 leader_election.cc:288] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:31.470727 31742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:31.470853 31742 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [term 1 LEADER]: Becoming Leader. State: Replica: f3ace0bf5567410ea16c36fb02155510, State: Running, Role: LEADER
I20260808 17:45:31.470938 31742 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ace0bf5567410ea16c36fb02155510" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37141 } }
I20260808 17:45:31.470989 31736 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:31.471267 31742 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3ace0bf5567410ea16c36fb02155510. Latest consensus state: current_term: 1 leader_uuid: "f3ace0bf5567410ea16c36fb02155510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ace0bf5567410ea16c36fb02155510" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37141 } } }
I20260808 17:45:31.471349 31742 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:31.471279 31747 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3ace0bf5567410ea16c36fb02155510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ace0bf5567410ea16c36fb02155510" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37141 } } }
I20260808 17:45:31.471531 31747 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:31.471580 31753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:31.472127 31753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:31.473428 31776 mvcc.cc:205] Tried to move back new op lower bound from 7316320794513625088 to 7316320794505342976. Current Snapshot: MvccSnapshot[applied={T|T < 7316320794513625088}]
I20260808 17:45:31.474022 31753 catalog_manager.cc:1347] Generated new cluster ID: 4cfe22c79cd24838a9ce43ebcd2ebd74
I20260808 17:45:31.474072 31753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:31.479748 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.468589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58492 (local address 127.29.62.254:37141)
0808 17:45:31.468777 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.468787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.468829 (+ 42us) server_negotiation.cc:407] Connection header received
0808 17:45:31.469066 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.469072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.469117 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.469229 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:31.470066 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.471364 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.472427 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.473113 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.473530 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.476035 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:31.476039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:31.476039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:31.476051 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:31.478048 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.478464 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.478467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.478468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.478623 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.478831 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.478832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.478832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.479143 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:31.479326 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.479393 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.479540 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260808 17:45:31.482417 31753 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:31.483119 31753 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:31.496006 31753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510: Generated new TSK 0
I20260808 17:45:31.496198 31753 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:31.528653 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:37141
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:31.611937 31782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:31.612138 31782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:31.612161 31782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:31.613349 31782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:31.613389 31782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:31.613410 31782 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:45:31.613427 31782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:31.614897 31782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:37141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:31.615084 31782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:31.615262 31782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:31.620107 31789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:31.620105 31787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:31.620239 31782 server_base.cc:1034] running on GCE node
W20260808 17:45:31.620105 31791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:31.620517 31782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:31.620787 31782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:31.621924 31782 hybrid_clock.cc:648] HybridClock initialized: now 1786211131621903 us; error 39 us; skew 500 ppm
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:31.624428 31782 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:31.624905 31782 webserver.cc:460] Webserver started at http://127.29.62.193:41633/ using document root <none> and password file <none>
I20260808 17:45:31.625146 31782 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:31.625216 31782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:31.625370 31782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:31.626521 31782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "ba533d036f50467fa511b05733be91aa"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "5917674bbe7cd07b48d0499e52fd9c52"
server_key_iv: "dce4dbb744ee8075f8e7601f3670f270"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.626891 31782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "ba533d036f50467fa511b05733be91aa"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "5917674bbe7cd07b48d0499e52fd9c52"
server_key_iv: "dce4dbb744ee8075f8e7601f3670f270"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.628221 31782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:31.628928 31800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:31.629099 31782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:31.629208 31782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "ba533d036f50467fa511b05733be91aa"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "5917674bbe7cd07b48d0499e52fd9c52"
server_key_iv: "dce4dbb744ee8075f8e7601f3670f270"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.629282 31782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:31.636327 31782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:31.638020 31782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:31.638175 31782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:31.638777 31782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:31.638844 31782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:31.638893 31782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:31.638921 31782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:31.646525 31782 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:43307
I20260808 17:45:31.646595 31974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:43307 every 8 connection(s)
I20260808 17:45:31.646948 31782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:31.648135 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.639154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:45707 (local address 127.29.62.254:37141)
0808 17:45:31.639226 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.639230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.639474 (+ 244us) server_negotiation.cc:407] Connection header received
0808 17:45:31.639923 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.639925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.639944 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.640030 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:31.640740 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.641276 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.642426 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.642671 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.642750 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.645082 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:31.645083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:31.645084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:31.645092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:31.646874 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.647348 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.647350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.647350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.647418 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.647645 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.647645 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.647646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.647752 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:31.647803 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.647999 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.648072 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:45:31.648288 31811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.639152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:37141 (local address 127.29.62.193:45707)
0808 17:45:31.639329 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.639368 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.639832 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.640095 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.640098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.640380 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:31.640622 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.640628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.641413 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.641414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.642316 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.642319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.642757 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.642758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.642889 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.643418 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:31.643426 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:31.644985 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:31.646954 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.646958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.646960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.647255 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.647484 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.647485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.647485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.647583 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.647794 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:31.647796 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:31.647896 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 17:45:31.647968 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:45:31.648091 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260808 17:45:31.648793 31975 heartbeater.cc:347] Connected to a master server at 127.29.62.254:37141
I20260808 17:45:31.648877 31975 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:31.649063 31975 heartbeater.cc:511] Master 127.29.62.254:37141 requested a full tablet report, sending...
I20260808 17:45:31.649641 31663 ts_manager.cc:194] Registered new tserver with Master: ba533d036f50467fa511b05733be91aa (127.29.62.193:43307)
I20260808 17:45:31.650326 31663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:45707
I20260808 17:45:31.653734 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31782
I20260808 17:45:31.653849 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:31.654038 29947 external_mini_cluster.cc:1428] Setting key 733d4d619456fa5162fa63b478d7b678
I20260808 17:45:31.655483 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.649887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:37369 (local address 127.29.62.254:37141)
0808 17:45:31.649958 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.649960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.649996 (+ 36us) server_negotiation.cc:407] Connection header received
0808 17:45:31.650067 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.650067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.650077 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.650121 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:31.650530 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.651090 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.651790 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.652000 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.652077 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.652972 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:31.652974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:31.652974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:31.652980 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:31.654329 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.654691 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.654692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.654692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.654749 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.654948 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.654949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.654949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.655054 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:31.655125 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.655308 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.655412 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:45:31.655773 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.649822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:37141 (local address 127.29.62.193:37369)
0808 17:45:31.649940 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.649952 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.650011 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.650123 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.650124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.650317 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:31.650365 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.650411 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.651195 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.651196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.651705 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.651707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.652075 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.652076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.652157 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.652588 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:31.652593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:31.652862 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:31.654452 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.654453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.654454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.654620 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.654819 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.654820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.654820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.654877 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.655130 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:31.655131 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:31.655192 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 17:45:31.655546 (+ 354us) client_negotiation.cc:240] Negotiation successful
0808 17:45:31.655681 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:31.678939 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:37141
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:31.760515 31986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:31.760735 31986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:31.760758 31986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:31.761826 31986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:31.761857 31986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:31.761878 31986 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:45:31.761894 31986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:31.763247 31986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:37141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:31.763427 31986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:31.763603 31986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:31.768905 31993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:31.768936 31992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:31.768913 31995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:31.769096 31986 server_base.cc:1034] running on GCE node
I20260808 17:45:31.769397 31986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:31.769609 31986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:31.770787 31986 hybrid_clock.cc:648] HybridClock initialized: now 1786211131770770 us; error 53 us; skew 500 ppm
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:31.773460 31986 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:31.773905 31986 webserver.cc:460] Webserver started at http://127.29.62.194:35901/ using document root <none> and password file <none>
I20260808 17:45:31.774096 31986 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:31.774135 31986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:31.774224 31986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:31.775292 31986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "6f84677e9d814065afeae235348c64b2"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "47cf8064f9b741bed248e55b74fcace6"
server_key_iv: "b392fe5c2bff79885a97e844086de96b"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.775614 31986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "6f84677e9d814065afeae235348c64b2"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "47cf8064f9b741bed248e55b74fcace6"
server_key_iv: "b392fe5c2bff79885a97e844086de96b"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.776876 31986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:31.777565 32004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:31.777721 31986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:31.777813 31986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "6f84677e9d814065afeae235348c64b2"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "47cf8064f9b741bed248e55b74fcace6"
server_key_iv: "b392fe5c2bff79885a97e844086de96b"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.777869 31986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:31.787250 31986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:31.789058 31986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:31.789171 31986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:31.789616 31986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:31.789655 31986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:31.789680 31986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:31.789728 31986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:31.796505 31986 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:41071
I20260808 17:45:31.796581 32205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:41071 every 8 connection(s)
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:31.796878 31986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:31.799808 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.790138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:43477 (local address 127.29.62.254:37141)
0808 17:45:31.790212 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.790214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.790454 (+ 240us) server_negotiation.cc:407] Connection header received
0808 17:45:31.790978 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.790980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.790993 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.791049 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:31.791919 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.792739 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.794338 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.794693 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.794784 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.797406 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:31.797407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:31.797407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:31.797414 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:31.798686 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.799065 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.799065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.799066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.799116 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.799319 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.799320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.799320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.799417 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:31.799465 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.799686 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.799746 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:45:31.799979 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.790105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:37141 (local address 127.29.62.194:43477)
0808 17:45:31.790304 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.790343 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.790869 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.791115 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.791120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.791517 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:31.791805 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.791811 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.792834 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.792835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.794214 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.794217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.794789 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.794790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.794892 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.795453 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:31.795464 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:31.797316 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:31.798753 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.798759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.798760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.799001 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.799173 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.799174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.799174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.799267 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.799470 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:31.799474 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:31.799577 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 17:45:31.799672 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 17:45:31.799786 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260808 17:45:31.800484 32206 heartbeater.cc:347] Connected to a master server at 127.29.62.254:37141
I20260808 17:45:31.800570 32206 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:31.800761 32206 heartbeater.cc:511] Master 127.29.62.254:37141 requested a full tablet report, sending...
I20260808 17:45:31.801162 31662 ts_manager.cc:194] Registered new tserver with Master: 6f84677e9d814065afeae235348c64b2 (127.29.62.194:41071)
I20260808 17:45:31.801664 31662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:43477
I20260808 17:45:31.803897 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31986
I20260808 17:45:31.804018 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:31.804222 29947 external_mini_cluster.cc:1428] Setting key 6de5aa4ed39d6b94f862cf715ed686cc
I20260808 17:45:31.807235 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.801468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:37141 (local address 127.29.62.194:42103)
0808 17:45:31.801605 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.801616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.801676 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.801837 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.801838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.802023 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:31.802071 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.802073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.802784 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.802785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.803292 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.803294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.803658 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.803659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.803735 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.804206 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:31.804211 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:31.804595 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:31.806082 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.806083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.806084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.806323 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.806567 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.806568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.806568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.806643 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.806999 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:31.807000 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:31.807039 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:45:31.807096 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 17:45:31.807173 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260808 17:45:31.807343 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.801557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:42103 (local address 127.29.62.254:37141)
0808 17:45:31.801601 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.801603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.801667 (+ 64us) server_negotiation.cc:407] Connection header received
0808 17:45:31.801749 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.801750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.801760 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.801813 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:31.802151 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.802686 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.803379 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.803583 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.803658 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.804694 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:31.804696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:31.804696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:31.804703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:31.806001 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.806440 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.806441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.806441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.806500 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.806712 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.806713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.806713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.806921 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:31.806993 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.807144 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.807228 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:31.828606 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:37141
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:31.911463 32219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:31.911659 32219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:31.911684 32219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:31.912833 32219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:31.912875 32219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:31.912896 32219 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:45:31.912954 32219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:31.914453 32219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:37141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:31.914691 32219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:31.914915 32219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:31.919822 32229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:31.919914 32219 server_base.cc:1034] running on GCE node
W20260808 17:45:31.919842 32227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:31.919823 32231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:31.920291 32219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:31.920538 32219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:31.921680 32219 hybrid_clock.cc:648] HybridClock initialized: now 1786211131921672 us; error 28 us; skew 500 ppm
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:31.924085 32219 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:31.924563 32219 webserver.cc:460] Webserver started at http://127.29.62.195:37825/ using document root <none> and password file <none>
I20260808 17:45:31.924803 32219 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:31.924872 32219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:31.925036 32219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:31.926074 32219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "5b8df666afaf41d98048260d34fd6ade"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "c40b9d07a9c896b2f55ab40ec17e7cb7"
server_key_iv: "f61964bce8c60fb1ef75698c5a0251d4"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.926507 32219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "5b8df666afaf41d98048260d34fd6ade"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "c40b9d07a9c896b2f55ab40ec17e7cb7"
server_key_iv: "f61964bce8c60fb1ef75698c5a0251d4"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.927865 32219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:31.928597 32242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:31.928758 32219 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:31.928857 32219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "5b8df666afaf41d98048260d34fd6ade"
format_stamp: "Formatted at 2026-08-08 17:45:31 on dist-test-slave-tk9z"
server_key: "c40b9d07a9c896b2f55ab40ec17e7cb7"
server_key_iv: "f61964bce8c60fb1ef75698c5a0251d4"
server_key_version: "encryptionkey@0"
I20260808 17:45:31.928961 32219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:31.947458 32219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:31.949564 32219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:31.949735 32219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:31.950460 32219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:31.950529 32219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:31.950578 32219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:31.950605 32219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:31.958540 32219 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:38157
I20260808 17:45:31.958921 32219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:45:31.958922 32427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:38157 every 8 connection(s)
I20260808 17:45:31.961614 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.951158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:50001 (local address 127.29.62.254:37141)
0808 17:45:31.951239 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.951241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.951489 (+ 248us) server_negotiation.cc:407] Connection header received
0808 17:45:31.952059 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.952060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.952075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.952129 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:31.952843 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.953482 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.954631 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.954851 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.954933 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.958323 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:31.958325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:31.958325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:31.958339 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:31.959969 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.960564 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.960566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.960566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.960639 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.960925 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.960926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.960926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.961063 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:31.961140 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.961434 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.961521 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 17:45:31.961798 32256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.951150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:37141 (local address 127.29.62.195:50001)
0808 17:45:31.951360 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.951397 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.951946 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.952208 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.952212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.952524 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:31.952725 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.952730 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.953576 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.953577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.954537 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.954540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.954942 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.954943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.955078 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.955645 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:31.955656 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:31.958221 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:31.960086 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.960092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.960094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.960458 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.960728 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.960729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.960729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.960864 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.961146 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:31.961153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:31.961293 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 17:45:31.961395 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 17:45:31.961571 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":110,"threads_started":1}
I20260808 17:45:31.962352 32429 heartbeater.cc:347] Connected to a master server at 127.29.62.254:37141
I20260808 17:45:31.962476 32429 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:31.962704 32429 heartbeater.cc:511] Master 127.29.62.254:37141 requested a full tablet report, sending...
I20260808 17:45:31.963493 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 32219
I20260808 17:45:31.963603 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:31.963709 31662 ts_manager.cc:194] Registered new tserver with Master: 5b8df666afaf41d98048260d34fd6ade (127.29.62.195:38157)
I20260808 17:45:31.963811 29947 external_mini_cluster.cc:1428] Setting key ee21b72d83e2bc98df709e24eb54569d
I20260808 17:45:31.964188 31662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:50001
I20260808 17:45:31.964709 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:45:31.970500 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.963843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:51461 (local address 127.29.62.254:37141)
0808 17:45:31.963883 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.963885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.964145 (+ 260us) server_negotiation.cc:407] Connection header received
0808 17:45:31.964233 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.964234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.964245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.964291 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:31.964773 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.965478 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.966567 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.966750 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.966811 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.967776 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:31.967777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:31.967778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:31.967786 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:31.969267 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.969676 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.969677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.969678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.969742 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.969961 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.969962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.969962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.970067 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:31.970119 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.970305 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.970362 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:45:31.970530 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.963779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:37141 (local address 127.29.62.195:51461)
0808 17:45:31.964084 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.964094 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.964171 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.964336 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.964337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.964559 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:31.964674 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.964677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.965598 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.965599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.966474 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.966477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.966832 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.966833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.966922 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.967409 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:31.967413 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:31.967690 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:31.969354 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.969355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.969356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.969597 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.969816 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:31.969816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:31.969817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:31.969894 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:31.970136 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:31.970137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:31.970192 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 17:45:31.970287 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 17:45:31.970426 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:45:31 dist-test-slave-tk9z krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:31.989900 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.982692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.29.62.254:37141)
0808 17:45:31.982756 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.982758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.982767 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:31.982813 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.982814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.982824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.982891 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:31.983432 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.983913 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.984632 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.984827 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.984912 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.987265 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:31.987267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:31.987273 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:31.987281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:31.988688 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.989093 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.989094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.989095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.989156 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:31.989376 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:31.989377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:31.989377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:31.989490 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:31.989596 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.989708 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.989792 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:45:31.991663 31662 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:31.993171 31662 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:31.999881 32455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.995853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.29.62.195:38157)
0808 17:45:31.996057 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.996063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.996123 (+ 60us) server_negotiation.cc:407] Connection header received
0808 17:45:31.996209 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.996212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.996227 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.996504 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:31.996571 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.997700 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.998904 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.999460 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.999557 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.999629 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:31.999746 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 17:45:31.999784 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260808 17:45:31.999874 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.995492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:38157 (local address 127.0.0.1:40962)
0808 17:45:31.996057 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.996068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.996130 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.996346 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.996347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.996350 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:31.996420 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.996422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.997825 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.997826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.998813 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.998816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.999558 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.999559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.999641 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:31.999655 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 17:45:31.999730 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 17:45:31.999779 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260808 17:45:32.000309 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.995491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:43307 (local address 127.0.0.1:55070)
0808 17:45:31.995673 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.995718 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.995784 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.996308 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.996311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.996313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:31.996391 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.996394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.997789 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.997790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.998946 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.998949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.999890 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.999891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.000151 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.000157 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.000194 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.000238 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":54,"thread_start_us":65,"threads_started":1}
I20260808 17:45:32.000543 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.995726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:41071 (local address 127.0.0.1:42954)
0808 17:45:31.996211 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:31.996218 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:31.996270 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:31.996661 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:31.996662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:31.996666 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:31.996700 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.996701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.997563 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.997565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:31.998962 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:31.998963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.000268 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.000269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.000327 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.000331 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.000376 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.000416 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":4,"thread_start_us":56,"threads_started":1}
I20260808 17:45:32.000802 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.995884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42954 (local address 127.29.62.194:41071)
0808 17:45:31.996094 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.996098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.996324 (+ 226us) server_negotiation.cc:407] Connection header received
0808 17:45:31.996372 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.996374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.996517 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.996596 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:31.996775 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.997469 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.999052 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.999785 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.999858 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.000620 (+ 762us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.000654 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.000702 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":107,"threads_started":1}
I20260808 17:45:32.000836 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:31.995633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.29.62.193:43307)
0808 17:45:31.996110 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:31.996114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:31.996123 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:31.996166 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:31.996168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:31.996186 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:31.996520 (+ 334us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:31.996581 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.997699 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:31.999051 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:31.999786 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.000524 (+ 738us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.000591 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.000689 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.000730 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":109,"threads_started":1}
I20260808 17:45:32.000758 32324 tablet_service.cc:1467] Processing CreateTablet for tablet 2e9297d926814071aa694a3035d7ed11 (DEFAULT_TABLE table=test-table [id=a7b45d3f035e4acd8e7b073286a7e0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:32.001176 32324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e9297d926814071aa694a3035d7ed11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.001420 32101 tablet_service.cc:1467] Processing CreateTablet for tablet 2e9297d926814071aa694a3035d7ed11 (DEFAULT_TABLE table=test-table [id=a7b45d3f035e4acd8e7b073286a7e0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:32.001412 31878 tablet_service.cc:1467] Processing CreateTablet for tablet 2e9297d926814071aa694a3035d7ed11 (DEFAULT_TABLE table=test-table [id=a7b45d3f035e4acd8e7b073286a7e0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:32.001770 31878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e9297d926814071aa694a3035d7ed11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.001770 32101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e9297d926814071aa694a3035d7ed11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.004093 32462 tablet_bootstrap.cc:492] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: Bootstrap starting.
I20260808 17:45:32.004875 32462 tablet_bootstrap.cc:654] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.005211 32462 log.cc:824] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:32.005802 32463 tablet_bootstrap.cc:492] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Bootstrap starting.
I20260808 17:45:32.006023 32462 tablet_bootstrap.cc:492] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: No bootstrap required, opened a new log
I20260808 17:45:32.006108 32462 ts_tablet_manager.cc:1397] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:32.006466 32463 tablet_bootstrap.cc:654] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.006554 32465 tablet_bootstrap.cc:492] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: Bootstrap starting.
I20260808 17:45:32.006834 32463 log.cc:824] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:32.007197 32465 tablet_bootstrap.cc:654] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.007519 32465 log.cc:824] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:32.007735 32463 tablet_bootstrap.cc:492] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: No bootstrap required, opened a new log
I20260808 17:45:32.007808 32463 ts_tablet_manager.cc:1397] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:45:32.008204 32462 raft_consensus.cc:348] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.008347 32462 raft_consensus.cc:374] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.008396 32462 raft_consensus.cc:729] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b8df666afaf41d98048260d34fd6ade, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.008396 32465 tablet_bootstrap.cc:492] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: No bootstrap required, opened a new log
I20260808 17:45:32.008533 32465 ts_tablet_manager.cc:1397] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:45:32.008520 32462 consensus_queue.cc:260] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.008873 32462 ts_tablet_manager.cc:1428] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:45:32.009006 32429 heartbeater.cc:503] Master 127.29.62.254:37141 was elected leader, sending a full tablet report...
I20260808 17:45:32.009863 32463 raft_consensus.cc:348] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.010042 32463 raft_consensus.cc:374] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.010079 32463 raft_consensus.cc:729] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f84677e9d814065afeae235348c64b2, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.010190 32463 consensus_queue.cc:260] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.010517 32463 ts_tablet_manager.cc:1428] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:45:32.010560 32206 heartbeater.cc:503] Master 127.29.62.254:37141 was elected leader, sending a full tablet report...
I20260808 17:45:32.010931 32465 raft_consensus.cc:348] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.011202 32465 raft_consensus.cc:374] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.011276 32465 raft_consensus.cc:729] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba533d036f50467fa511b05733be91aa, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.011454 32465 consensus_queue.cc:260] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.011714 32465 ts_tablet_manager.cc:1428] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:45:32.011776 31975 heartbeater.cc:503] Master 127.29.62.254:37141 was elected leader, sending a full tablet report...
W20260808 17:45:32.047645 32207 tablet.cc:2369] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:45:32.048069 32468 raft_consensus.cc:482] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:32.048245 32468 raft_consensus.cc:504] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.048584 32468 leader_election.cc:288] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba533d036f50467fa511b05733be91aa (127.29.62.193:43307), 6f84677e9d814065afeae235348c64b2 (127.29.62.194:41071)
I20260808 17:45:32.051517 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.048830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:46861 (local address 127.29.62.193:43307)
0808 17:45:32.048895 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.048897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.048907 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:32.048941 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.048942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.048953 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.049056 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:32.049233 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.049769 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.050784 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.051151 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.051245 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.051254 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.051418 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.051454 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:45:32.051517 32256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.048755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:43307 (local address 127.29.62.195:46861)
0808 17:45:32.048816 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.048828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.048875 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.049066 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.049068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.049070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:32.049147 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.049150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.049853 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.049854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.050695 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.050697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.051244 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.051245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.051337 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.051340 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.051405 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.051445 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 17:45:32.051597 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.048998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:59361 (local address 127.29.62.194:41071)
0808 17:45:32.049095 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.049097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.049136 (+ 39us) server_negotiation.cc:407] Connection header received
0808 17:45:32.049202 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.049203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.049212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.049306 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:32.049482 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.049946 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.050885 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.051226 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.051324 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.051332 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.051491 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.051527 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 17:45:32.051651 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.048896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:41071 (local address 127.29.62.195:59361)
0808 17:45:32.049082 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.049092 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.049149 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.049311 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.049313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.049314 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:32.049403 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.049405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.050051 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.050052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.050794 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.050796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.051325 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.051326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.051408 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.051411 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.051481 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.051523 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":124,"threads_started":1}
I20260808 17:45:32.051910 31909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e9297d926814071aa694a3035d7ed11" candidate_uuid: "5b8df666afaf41d98048260d34fd6ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba533d036f50467fa511b05733be91aa" is_pre_election: true
I20260808 17:45:32.051975 32140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e9297d926814071aa694a3035d7ed11" candidate_uuid: "5b8df666afaf41d98048260d34fd6ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f84677e9d814065afeae235348c64b2" is_pre_election: true
I20260808 17:45:32.052085 31909 raft_consensus.cc:2454] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b8df666afaf41d98048260d34fd6ade in term 0.
I20260808 17:45:32.052085 32140 raft_consensus.cc:2454] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b8df666afaf41d98048260d34fd6ade in term 0.
I20260808 17:45:32.052326 32245 leader_election.cc:302] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b8df666afaf41d98048260d34fd6ade, 6f84677e9d814065afeae235348c64b2; no voters:
I20260808 17:45:32.052453 32468 raft_consensus.cc:2781] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:32.052512 32468 raft_consensus.cc:482] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:32.052536 32468 raft_consensus.cc:3037] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.053270 32468 raft_consensus.cc:504] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.053408 32468 leader_election.cc:288] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [CANDIDATE]: Term 1 election: Requested vote from peers ba533d036f50467fa511b05733be91aa (127.29.62.193:43307), 6f84677e9d814065afeae235348c64b2 (127.29.62.194:41071)
I20260808 17:45:32.053603 31909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e9297d926814071aa694a3035d7ed11" candidate_uuid: "5b8df666afaf41d98048260d34fd6ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba533d036f50467fa511b05733be91aa"
I20260808 17:45:32.053603 32140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e9297d926814071aa694a3035d7ed11" candidate_uuid: "5b8df666afaf41d98048260d34fd6ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f84677e9d814065afeae235348c64b2"
I20260808 17:45:32.053684 32140 raft_consensus.cc:3037] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.053685 31909 raft_consensus.cc:3037] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.054610 32140 raft_consensus.cc:2454] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b8df666afaf41d98048260d34fd6ade in term 1.
I20260808 17:45:32.054610 31909 raft_consensus.cc:2454] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b8df666afaf41d98048260d34fd6ade in term 1.
I20260808 17:45:32.054802 32245 leader_election.cc:302] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b8df666afaf41d98048260d34fd6ade, 6f84677e9d814065afeae235348c64b2; no voters:
I20260808 17:45:32.054890 32468 raft_consensus.cc:2781] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:32.055083 32468 raft_consensus.cc:686] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 1 LEADER]: Becoming Leader. State: Replica: 5b8df666afaf41d98048260d34fd6ade, State: Running, Role: LEADER
I20260808 17:45:32.055205 32468 consensus_queue.cc:237] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } }
I20260808 17:45:32.056133 31662 catalog_manager.cc:5495] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b8df666afaf41d98048260d34fd6ade (127.29.62.195). New cstate: current_term: 1 leader_uuid: "5b8df666afaf41d98048260d34fd6ade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } health_report { overall_health: UNKNOWN } } }
I20260808 17:45:32.076370 32455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.073574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.29.62.195:38157)
0808 17:45:32.073656 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.073658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.073668 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:32.073741 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.073743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.073767 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.073837 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:32.074004 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.074530 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.075322 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.075519 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.075595 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.075821 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:32.075926 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:32.076016 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.076072 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.076097 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:45:32.078298 31909 raft_consensus.cc:1261] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Refusing update from remote peer 5b8df666afaf41d98048260d34fd6ade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:32.078428 32140 raft_consensus.cc:1261] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Refusing update from remote peer 5b8df666afaf41d98048260d34fd6ade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:32.078754 32474 consensus_queue.cc:1035] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:32.078887 32468 consensus_queue.cc:1035] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:32.081180 32479 mvcc.cc:205] Tried to move back new op lower bound from 7316320796989665280 to 7316320796898803712. Current Snapshot: MvccSnapshot[applied={T|T < 7316320796989665280}]
I20260808 17:45:32.081487 32480 mvcc.cc:205] Tried to move back new op lower bound from 7316320796989665280 to 7316320796898803712. Current Snapshot: MvccSnapshot[applied={T|T < 7316320796989665280}]
I20260808 17:45:32.081808 32481 mvcc.cc:205] Tried to move back new op lower bound from 7316320796989665280 to 7316320796898803712. Current Snapshot: MvccSnapshot[applied={T|T < 7316320796989665280}]
I20260808 17:45:32.085247 31662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:32.085497 31662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:32.086416 31662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510: Sending DeleteTablet for 3 replicas of tablet 2e9297d926814071aa694a3035d7ed11
I20260808 17:45:32.086774 32324 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e9297d926814071aa694a3035d7ed11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:40962
I20260808 17:45:32.086830 32101 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e9297d926814071aa694a3035d7ed11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:42954
I20260808 17:45:32.086836 31878 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e9297d926814071aa694a3035d7ed11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:55070
I20260808 17:45:32.087265 31662 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:32.087422 31662 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:32.088688 31878 tablet_service.cc:1467] Processing CreateTablet for tablet a566086c59c541a8885cb2df4ec15c12 (DEFAULT_TABLE table=test-table [id=06402a615dd64c7cb8cb8c2ae2ff351a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:32.088688 32324 tablet_service.cc:1467] Processing CreateTablet for tablet a566086c59c541a8885cb2df4ec15c12 (DEFAULT_TABLE table=test-table [id=06402a615dd64c7cb8cb8c2ae2ff351a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:32.088841 32324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a566086c59c541a8885cb2df4ec15c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.088846 31878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a566086c59c541a8885cb2df4ec15c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.090144 32465 tablet_bootstrap.cc:492] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: Bootstrap starting.
I20260808 17:45:32.090312 32462 tablet_bootstrap.cc:492] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: Bootstrap starting.
I20260808 17:45:32.090641 32465 tablet_bootstrap.cc:654] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.090756 32462 tablet_bootstrap.cc:654] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.091356 32465 tablet_bootstrap.cc:492] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: No bootstrap required, opened a new log
I20260808 17:45:32.091423 32465 ts_tablet_manager.cc:1397] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:45:32.091545 32101 tablet_service.cc:1467] Processing CreateTablet for tablet a566086c59c541a8885cb2df4ec15c12 (DEFAULT_TABLE table=test-table [id=06402a615dd64c7cb8cb8c2ae2ff351a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:32.091598 32465 raft_consensus.cc:348] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.091691 32465 raft_consensus.cc:374] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.091691 32101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a566086c59c541a8885cb2df4ec15c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.091718 32465 raft_consensus.cc:729] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba533d036f50467fa511b05733be91aa, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.091768 32465 consensus_queue.cc:260] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.091876 32465 ts_tablet_manager.cc:1428] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 17:45:32.093032 32463 tablet_bootstrap.cc:492] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: Bootstrap starting.
I20260808 17:45:32.093467 32463 tablet_bootstrap.cc:654] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.093484 32462 tablet_bootstrap.cc:492] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: No bootstrap required, opened a new log
I20260808 17:45:32.093532 32462 ts_tablet_manager.cc:1397] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 17:45:32.093684 32462 raft_consensus.cc:348] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.093760 32462 raft_consensus.cc:374] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.093782 32462 raft_consensus.cc:729] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b8df666afaf41d98048260d34fd6ade, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.093825 32462 consensus_queue.cc:260] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.093919 32462 ts_tablet_manager.cc:1428] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:32.094463 32463 tablet_bootstrap.cc:492] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: No bootstrap required, opened a new log
I20260808 17:45:32.094485 32497 tablet_replica.cc:330] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: stopping tablet replica
I20260808 17:45:32.094516 32463 ts_tablet_manager.cc:1397] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:45:32.094601 32497 raft_consensus.cc:2229] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:32.094707 32497 raft_consensus.cc:2258] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:32.094671 32463 raft_consensus.cc:348] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.094725 32463 raft_consensus.cc:374] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.094739 32463 raft_consensus.cc:729] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f84677e9d814065afeae235348c64b2, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.094805 32463 consensus_queue.cc:260] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.094871 32501 tablet_replica.cc:330] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: stopping tablet replica
I20260808 17:45:32.094897 32463 ts_tablet_manager.cc:1428] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:32.094987 32501 raft_consensus.cc:2229] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:32.095084 32501 raft_consensus.cc:2258] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:32.095505 32501 ts_tablet_manager.cc:1905] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:32.095500 32497 ts_tablet_manager.cc:1905] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:32.097143 32497 ts_tablet_manager.cc:1918] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:32.097216 32497 log.cc:1196] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/2e9297d926814071aa694a3035d7ed11
I20260808 17:45:32.097209 32501 ts_tablet_manager.cc:1918] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:32.097282 32501 log.cc:1196] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/2e9297d926814071aa694a3035d7ed11
I20260808 17:45:32.097482 32497 ts_tablet_manager.cc:1939] T 2e9297d926814071aa694a3035d7ed11 P 5b8df666afaf41d98048260d34fd6ade: Deleting consensus metadata
I20260808 17:45:32.097571 32501 ts_tablet_manager.cc:1939] T 2e9297d926814071aa694a3035d7ed11 P ba533d036f50467fa511b05733be91aa: Deleting consensus metadata
I20260808 17:45:32.097898 31645 catalog_manager.cc:4841] TS 5b8df666afaf41d98048260d34fd6ade (127.29.62.195:38157): tablet 2e9297d926814071aa694a3035d7ed11 (table test-table [id=a7b45d3f035e4acd8e7b073286a7e0c4]) successfully deleted
I20260808 17:45:32.097970 31648 catalog_manager.cc:4841] TS ba533d036f50467fa511b05733be91aa (127.29.62.193:43307): tablet 2e9297d926814071aa694a3035d7ed11 (table test-table [id=a7b45d3f035e4acd8e7b073286a7e0c4]) successfully deleted
I20260808 17:45:32.098502 32498 tablet_replica.cc:330] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: stopping tablet replica
I20260808 17:45:32.098624 32498 raft_consensus.cc:2229] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:32.098731 32498 raft_consensus.cc:2258] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:32.099141 32498 ts_tablet_manager.cc:1905] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:32.100346 32498 ts_tablet_manager.cc:1918] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:32.100416 32498 log.cc:1196] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/2e9297d926814071aa694a3035d7ed11
I20260808 17:45:32.100718 32498 ts_tablet_manager.cc:1939] T 2e9297d926814071aa694a3035d7ed11 P 6f84677e9d814065afeae235348c64b2: Deleting consensus metadata
I20260808 17:45:32.101104 31648 catalog_manager.cc:4841] TS 6f84677e9d814065afeae235348c64b2 (127.29.62.194:41071): tablet 2e9297d926814071aa694a3035d7ed11 (table test-table [id=a7b45d3f035e4acd8e7b073286a7e0c4]) successfully deleted
I20260808 17:45:32.136178 32470 raft_consensus.cc:482] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:32.136298 32470 raft_consensus.cc:504] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.136663 32470 leader_election.cc:288] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba533d036f50467fa511b05733be91aa (127.29.62.193:43307), 5b8df666afaf41d98048260d34fd6ade (127.29.62.195:38157)
I20260808 17:45:32.140064 32455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.136984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:39663 (local address 127.29.62.195:38157)
0808 17:45:32.137033 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.137035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.137170 (+ 135us) server_negotiation.cc:407] Connection header received
0808 17:45:32.137243 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.137244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.137254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.137304 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:32.137517 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.138207 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.139280 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.139711 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.139786 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.139796 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.139952 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.139988 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:45:32.140127 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.136842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:38157 (local address 127.29.62.194:39663)
0808 17:45:32.137102 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.137108 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.137173 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.137354 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.137355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.137358 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:32.137444 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.137446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.138321 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.138321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.139178 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.139181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.139797 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.139798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.139875 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.139880 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.139939 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.139988 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":63,"threads_started":1}
I20260808 17:45:32.140553 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.136825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:43307 (local address 127.29.62.194:40275)
0808 17:45:32.136949 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.136963 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.137012 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.137292 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.137293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.137295 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:32.137365 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.137367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.138301 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.138302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.139534 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.139537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.140291 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.140293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.140383 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.140387 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.140442 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.140478 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":16}
I20260808 17:45:32.140506 32353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a566086c59c541a8885cb2df4ec15c12" candidate_uuid: "6f84677e9d814065afeae235348c64b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8df666afaf41d98048260d34fd6ade" is_pre_election: true
I20260808 17:45:32.140645 32353 raft_consensus.cc:2454] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f84677e9d814065afeae235348c64b2 in term 0.
I20260808 17:45:32.140730 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.137101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:40275 (local address 127.29.62.193:43307)
0808 17:45:32.137171 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.137173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.137183 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:32.137214 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.137215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.137225 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.137280 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:32.137450 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.138209 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.139636 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.140192 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.140281 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.140292 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.140615 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.140661 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:45:32.140841 32005 leader_election.cc:302] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b8df666afaf41d98048260d34fd6ade, 6f84677e9d814065afeae235348c64b2; no voters:
I20260808 17:45:32.140980 32470 raft_consensus.cc:2781] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:32.141026 32470 raft_consensus.cc:482] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:32.141060 32470 raft_consensus.cc:3037] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.141785 32470 raft_consensus.cc:504] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.141978 32470 leader_election.cc:288] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [CANDIDATE]: Term 1 election: Requested vote from peers ba533d036f50467fa511b05733be91aa (127.29.62.193:43307), 5b8df666afaf41d98048260d34fd6ade (127.29.62.195:38157)
I20260808 17:45:32.142079 31909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a566086c59c541a8885cb2df4ec15c12" candidate_uuid: "6f84677e9d814065afeae235348c64b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba533d036f50467fa511b05733be91aa" is_pre_election: true
I20260808 17:45:32.142127 31907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a566086c59c541a8885cb2df4ec15c12" candidate_uuid: "6f84677e9d814065afeae235348c64b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba533d036f50467fa511b05733be91aa"
I20260808 17:45:32.142180 31909 raft_consensus.cc:2454] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f84677e9d814065afeae235348c64b2 in term 0.
I20260808 17:45:32.142513 32353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a566086c59c541a8885cb2df4ec15c12" candidate_uuid: "6f84677e9d814065afeae235348c64b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8df666afaf41d98048260d34fd6ade"
I20260808 17:45:32.142594 32353 raft_consensus.cc:3037] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.143149 32353 raft_consensus.cc:2454] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f84677e9d814065afeae235348c64b2 in term 1.
I20260808 17:45:32.143337 32005 leader_election.cc:302] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b8df666afaf41d98048260d34fd6ade, 6f84677e9d814065afeae235348c64b2; no voters: ba533d036f50467fa511b05733be91aa
I20260808 17:45:32.143443 32470 raft_consensus.cc:2781] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:32.143579 32470 raft_consensus.cc:686] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 1 LEADER]: Becoming Leader. State: Replica: 6f84677e9d814065afeae235348c64b2, State: Running, Role: LEADER
I20260808 17:45:32.143651 32470 consensus_queue.cc:237] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.144371 31662 catalog_manager.cc:5495] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f84677e9d814065afeae235348c64b2 (127.29.62.194). New cstate: current_term: 1 leader_uuid: "6f84677e9d814065afeae235348c64b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } health_report { overall_health: UNKNOWN } } }
W20260808 17:45:32.147794 31976 tablet.cc:2369] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 17:45:32 dist-test-slave-tk9z krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:32.171141 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.163135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:37141 (local address 127.0.0.1:58500)
0808 17:45:32.163279 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.163290 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.163359 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.163498 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.163499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.163788 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:32.163868 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.163874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.164794 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.164795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.165315 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.165317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.165796 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.165797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.165848 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.166488 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:32.166493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:32.168104 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:32.169891 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:32.169898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:32.169899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:32.170094 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:32.170351 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:32.170352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:32.170352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:32.170468 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:32.170799 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:32.170806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:32.170849 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.170927 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.171048 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260808 17:45:32.171250 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.163199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58500 (local address 127.29.62.254:37141)
0808 17:45:32.163255 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.163257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.163368 (+ 111us) server_negotiation.cc:407] Connection header received
0808 17:45:32.163400 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.163401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.163414 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.163492 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:32.163952 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.164711 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.165405 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.165692 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.165774 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.168201 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:32.168203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:32.168203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:32.168211 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:32.169789 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:32.170179 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:32.170181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:32.170181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:32.170274 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:32.170541 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:32.170542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:32.170542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:32.170684 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:32.170764 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.171045 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.171136 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:45:32.173300 31662 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:58500:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 17:45:32.173532 31662 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:32.175113 32324 tablet_service.cc:1467] Processing CreateTablet for tablet 1921e8ca5a8349dca29441887e4a1df4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=19568e2cd869490f89a8cd3c0d7b4f34]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:45:32.175112 31878 tablet_service.cc:1467] Processing CreateTablet for tablet 1921e8ca5a8349dca29441887e4a1df4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=19568e2cd869490f89a8cd3c0d7b4f34]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:45:32.175164 32101 tablet_service.cc:1467] Processing CreateTablet for tablet 1921e8ca5a8349dca29441887e4a1df4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=19568e2cd869490f89a8cd3c0d7b4f34]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:45:32.175292 32324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1921e8ca5a8349dca29441887e4a1df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.175302 31878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1921e8ca5a8349dca29441887e4a1df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.175330 32101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1921e8ca5a8349dca29441887e4a1df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:32.176929 32465 tablet_bootstrap.cc:492] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: Bootstrap starting.
I20260808 17:45:32.176994 32462 tablet_bootstrap.cc:492] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: Bootstrap starting.
I20260808 17:45:32.177387 32465 tablet_bootstrap.cc:654] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.177469 32462 tablet_bootstrap.cc:654] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.178026 32463 tablet_bootstrap.cc:492] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: Bootstrap starting.
I20260808 17:45:32.178469 32463 tablet_bootstrap.cc:654] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:32.178646 32465 tablet_bootstrap.cc:492] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: No bootstrap required, opened a new log
I20260808 17:45:32.178681 32465 ts_tablet_manager.cc:1397] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:45:32.178839 32465 raft_consensus.cc:348] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.178920 32465 raft_consensus.cc:374] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.178946 32465 raft_consensus.cc:729] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba533d036f50467fa511b05733be91aa, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.178996 32465 consensus_queue.cc:260] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.179046 32462 tablet_bootstrap.cc:492] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: No bootstrap required, opened a new log
I20260808 17:45:32.179091 32462 ts_tablet_manager.cc:1397] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:32.179104 32465 ts_tablet_manager.cc:1428] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:32.179150 32471 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.179236 32471 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:32.179234 32462 raft_consensus.cc:348] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.179324 32462 raft_consensus.cc:374] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.179348 32462 raft_consensus.cc:729] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b8df666afaf41d98048260d34fd6ade, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.179396 32462 consensus_queue.cc:260] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.179483 32462 ts_tablet_manager.cc:1428] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:32.179509 32474 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.179615 32474 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:32.179886 32463 tablet_bootstrap.cc:492] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: No bootstrap required, opened a new log
I20260808 17:45:32.179943 32463 ts_tablet_manager.cc:1397] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:45:32.180096 32463 raft_consensus.cc:348] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.180174 32463 raft_consensus.cc:374] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:32.180197 32463 raft_consensus.cc:729] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f84677e9d814065afeae235348c64b2, State: Initialized, Role: FOLLOWER
I20260808 17:45:32.180248 32463 consensus_queue.cc:260] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.180346 32463 ts_tablet_manager.cc:1428] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:32.180420 32470 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.180516 32470 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 17:45:32.209721 32430 tablet.cc:2369] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:45:32.504695 32471 raft_consensus.cc:482] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:32.504823 32471 raft_consensus.cc:504] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.505203 32471 leader_election.cc:288] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f84677e9d814065afeae235348c64b2 (127.29.62.194:41071), 5b8df666afaf41d98048260d34fd6ade (127.29.62.195:38157)
I20260808 17:45:32.508312 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.505512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:52929 (local address 127.29.62.194:41071)
0808 17:45:32.505558 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.505560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.505663 (+ 103us) server_negotiation.cc:407] Connection header received
0808 17:45:32.505739 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.505740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.505748 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.505822 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:32.506012 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.506549 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.507576 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.507923 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.508025 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.508035 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.508201 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.508240 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:45:32.508337 32455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.505487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:60699 (local address 127.29.62.195:38157)
0808 17:45:32.505556 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.505558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.505658 (+ 100us) server_negotiation.cc:407] Connection header received
0808 17:45:32.505737 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.505738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.505764 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.505829 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:32.506027 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.506534 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.507584 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.507944 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.508028 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.508037 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.508226 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.508268 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:45:32.508368 32524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.505431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:41071 (local address 127.29.62.193:52929)
0808 17:45:32.505578 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.505589 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.505676 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.505826 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.505827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.505829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:32.505931 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.505933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.506630 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.506631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.507480 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.507483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.508044 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.508045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.508122 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.508126 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.508192 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.508232 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":37,"thread_start_us":61,"threads_started":1}
I20260808 17:45:32.508378 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.505423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:38157 (local address 127.29.62.193:60699)
0808 17:45:32.505574 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.505585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.505676 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.505831 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.505832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.505835 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:32.505944 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.505946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.506612 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.506613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.507485 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.507488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.508052 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.508054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.508131 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.508135 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.508217 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.508268 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":61,"thread_start_us":76,"threads_started":1}
I20260808 17:45:32.508653 32140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1921e8ca5a8349dca29441887e4a1df4" candidate_uuid: "ba533d036f50467fa511b05733be91aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f84677e9d814065afeae235348c64b2" is_pre_election: true
I20260808 17:45:32.508777 32140 raft_consensus.cc:2454] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba533d036f50467fa511b05733be91aa in term 0.
I20260808 17:45:32.508917 32353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1921e8ca5a8349dca29441887e4a1df4" candidate_uuid: "ba533d036f50467fa511b05733be91aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8df666afaf41d98048260d34fd6ade" is_pre_election: true
I20260808 17:45:32.508965 31803 leader_election.cc:302] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f84677e9d814065afeae235348c64b2, ba533d036f50467fa511b05733be91aa; no voters:
I20260808 17:45:32.509029 32353 raft_consensus.cc:2454] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba533d036f50467fa511b05733be91aa in term 0.
I20260808 17:45:32.509085 32471 raft_consensus.cc:2781] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:32.509124 32471 raft_consensus.cc:482] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:32.509146 32471 raft_consensus.cc:3037] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.509817 32471 raft_consensus.cc:504] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.509960 32471 leader_election.cc:288] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [CANDIDATE]: Term 1 election: Requested vote from peers 6f84677e9d814065afeae235348c64b2 (127.29.62.194:41071), 5b8df666afaf41d98048260d34fd6ade (127.29.62.195:38157)
I20260808 17:45:32.510156 32353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1921e8ca5a8349dca29441887e4a1df4" candidate_uuid: "ba533d036f50467fa511b05733be91aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8df666afaf41d98048260d34fd6ade"
I20260808 17:45:32.510169 32140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1921e8ca5a8349dca29441887e4a1df4" candidate_uuid: "ba533d036f50467fa511b05733be91aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f84677e9d814065afeae235348c64b2"
I20260808 17:45:32.510241 32353 raft_consensus.cc:3037] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.510241 32140 raft_consensus.cc:3037] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.511035 32140 raft_consensus.cc:2454] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba533d036f50467fa511b05733be91aa in term 1.
I20260808 17:45:32.511034 32353 raft_consensus.cc:2454] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba533d036f50467fa511b05733be91aa in term 1.
I20260808 17:45:32.511246 31801 leader_election.cc:302] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b8df666afaf41d98048260d34fd6ade, ba533d036f50467fa511b05733be91aa; no voters:
I20260808 17:45:32.511346 32471 raft_consensus.cc:2781] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:32.511498 32471 raft_consensus.cc:686] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [term 1 LEADER]: Becoming Leader. State: Replica: ba533d036f50467fa511b05733be91aa, State: Running, Role: LEADER
I20260808 17:45:32.511574 32471 consensus_queue.cc:237] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } }
I20260808 17:45:32.511940 32527 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: TxnStatusTablet state changed. Reason: New leader ba533d036f50467fa511b05733be91aa. Latest consensus state: current_term: 1 leader_uuid: "ba533d036f50467fa511b05733be91aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.512022 32527 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:45:32.512203 32529 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 17:45:32.512332 32529 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 17:45:32.512359 31663 catalog_manager.cc:5495] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa reported cstate change: term changed from 0 to 1, leader changed from <none> to ba533d036f50467fa511b05733be91aa (127.29.62.193). New cstate: current_term: 1 leader_uuid: "ba533d036f50467fa511b05733be91aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } health_report { overall_health: UNKNOWN } } }
I20260808 17:45:32.537335 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.534475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:43307 (local address 127.0.0.1:55076)
0808 17:45:32.534537 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.534549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.534612 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.534817 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.534818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.534819 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:45:32.534867 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.534869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.535781 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.535782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.536338 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.536340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.536827 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.536828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.536877 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.536891 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:45:32.537195 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:45:32.537202 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.537237 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.537277 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 17:45:32.537351 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.534493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55076 (local address 127.29.62.193:43307)
0808 17:45:32.534541 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.534543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.534608 (+ 65us) server_negotiation.cc:407] Connection header received
0808 17:45:32.534698 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.534699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.534721 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.534808 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:32.534940 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.535702 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.536428 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.536745 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.536817 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.537002 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:32.537126 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:32.537201 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.537247 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.537270 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:45:32.538811 32140 raft_consensus.cc:1261] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Refusing update from remote peer ba533d036f50467fa511b05733be91aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:32.538941 32353 raft_consensus.cc:1261] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Refusing update from remote peer ba533d036f50467fa511b05733be91aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:32.539268 32527 consensus_queue.cc:1035] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:32.539441 32471 consensus_queue.cc:1035] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:32.540202 32474 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: TxnStatusTablet state changed. Reason: New leader ba533d036f50467fa511b05733be91aa. Latest consensus state: current_term: 1 leader_uuid: "ba533d036f50467fa511b05733be91aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.540242 32470 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: TxnStatusTablet state changed. Reason: New leader ba533d036f50467fa511b05733be91aa. Latest consensus state: current_term: 1 leader_uuid: "ba533d036f50467fa511b05733be91aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.540335 32470 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:32.540330 32474 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:32.541204 32471 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba533d036f50467fa511b05733be91aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.541285 32471 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:45:32.541560 32474 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba533d036f50467fa511b05733be91aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.541642 32474 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P 5b8df666afaf41d98048260d34fd6ade: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:32.542150 32471 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba533d036f50467fa511b05733be91aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.542235 32471 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P ba533d036f50467fa511b05733be91aa: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:45:32.542520 32470 tablet_replica.cc:439] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba533d036f50467fa511b05733be91aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f84677e9d814065afeae235348c64b2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 41071 } } peers { permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 } } peers { permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 } } }
I20260808 17:45:32.542608 32470 tablet_replica.cc:442] T 1921e8ca5a8349dca29441887e4a1df4 P 6f84677e9d814065afeae235348c64b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:32.546154 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.543644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.29.62.194:41071)
0808 17:45:32.543688 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.543690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.543699 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:32.543777 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.543778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.543798 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.543874 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:32.543951 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.544684 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.545350 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.545658 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.545728 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.545830 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:32.545950 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:32.546019 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.546042 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.546061 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:45:32.551581 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.548724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:49983 (local address 127.29.62.193:43307)
0808 17:45:32.548793 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.548813 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.548901 (+ 88us) server_negotiation.cc:407] Connection header received
0808 17:45:32.548969 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.548969 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.548977 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.549056 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:32.549198 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.549790 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.550921 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.551103 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.551157 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.551357 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:32.551381 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:32.551431 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.551503 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.551521 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 17:45:32.551635 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.548641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:43307 (local address 127.29.62.194:49983)
0808 17:45:32.548835 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.548846 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.548912 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.549062 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.549062 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.549063 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:45:32.549126 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.549128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.549859 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.549860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.550834 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.550837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.551179 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.551180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.551259 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.551274 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:45:32.551437 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:45:32.551439 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.551500 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.551547 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260808 17:45:32.554286 32353 raft_consensus.cc:1261] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Refusing update from remote peer 6f84677e9d814065afeae235348c64b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:32.554580 31907 raft_consensus.cc:3037] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:32.554768 32506 consensus_queue.cc:1035] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b8df666afaf41d98048260d34fd6ade" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 38157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:32.555557 31907 raft_consensus.cc:1261] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Refusing update from remote peer 6f84677e9d814065afeae235348c64b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:32.556839 32506 consensus_queue.cc:1035] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba533d036f50467fa511b05733be91aa" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 43307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:32.570664 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.567834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:53733 (local address 127.29.62.194:41071)
0808 17:45:32.567884 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.567886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.568056 (+ 170us) server_negotiation.cc:407] Connection header received
0808 17:45:32.568121 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.568121 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.568130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.568172 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:32.568306 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.568757 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.569896 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.570074 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.570119 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.570328 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:32.570354 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:32.570466 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.570558 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.570584 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:45:32.570685 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.567777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:41071 (local address 127.29.62.193:53733)
0808 17:45:32.567989 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:32.568001 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:32.568063 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:32.568175 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:32.568176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:32.568188 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:45:32.568251 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.568253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.568831 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.568832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.569806 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:32.569809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.570144 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.570145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:32.570217 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.570235 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:45:32.570483 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:45:32.570486 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:32.570540 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:45:32.570587 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
I20260808 17:45:32.581748 31662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:32.581861 31662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:32.582634 31662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3ace0bf5567410ea16c36fb02155510: Sending DeleteTablet for 3 replicas of tablet a566086c59c541a8885cb2df4ec15c12
I20260808 17:45:32.582952 32324 tablet_service.cc:1514] Processing DeleteTablet for tablet a566086c59c541a8885cb2df4ec15c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:40962
I20260808 17:45:32.583024 31878 tablet_service.cc:1514] Processing DeleteTablet for tablet a566086c59c541a8885cb2df4ec15c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:55070
I20260808 17:45:32.583071 32497 tablet_replica.cc:330] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: stopping tablet replica
I20260808 17:45:32.583140 32497 raft_consensus.cc:2229] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:32.583144 32501 tablet_replica.cc:330] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: stopping tablet replica
I20260808 17:45:32.583194 32497 raft_consensus.cc:2258] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:32.583204 32501 raft_consensus.cc:2229] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:32.583256 32501 raft_consensus.cc:2258] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:32.583752 32501 ts_tablet_manager.cc:1905] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:32.585440 32501 ts_tablet_manager.cc:1918] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:32.585510 32501 log.cc:1196] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/a566086c59c541a8885cb2df4ec15c12
I20260808 17:45:32.585733 32501 ts_tablet_manager.cc:1939] T a566086c59c541a8885cb2df4ec15c12 P ba533d036f50467fa511b05733be91aa: Deleting consensus metadata
I20260808 17:45:32.586088 31648 catalog_manager.cc:4841] TS ba533d036f50467fa511b05733be91aa (127.29.62.193:43307): tablet a566086c59c541a8885cb2df4ec15c12 (table test-table [id=06402a615dd64c7cb8cb8c2ae2ff351a]) successfully deleted
I20260808 17:45:32.586241 32101 tablet_service.cc:1514] Processing DeleteTablet for tablet a566086c59c541a8885cb2df4ec15c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:42954
I20260808 17:45:32.586336 32498 tablet_replica.cc:330] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: stopping tablet replica
I20260808 17:45:32.586437 32498 raft_consensus.cc:2229] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:32.586527 32498 raft_consensus.cc:2258] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:32.586853 32498 ts_tablet_manager.cc:1905] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:32.588380 32498 ts_tablet_manager.cc:1918] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:32.588454 32498 log.cc:1196] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/a566086c59c541a8885cb2df4ec15c12
I20260808 17:45:32.588671 32498 ts_tablet_manager.cc:1939] T a566086c59c541a8885cb2df4ec15c12 P 6f84677e9d814065afeae235348c64b2: Deleting consensus metadata
I20260808 17:45:32.588989 31648 catalog_manager.cc:4841] TS 6f84677e9d814065afeae235348c64b2 (127.29.62.194:41071): tablet a566086c59c541a8885cb2df4ec15c12 (table test-table [id=06402a615dd64c7cb8cb8c2ae2ff351a]) successfully deleted
I20260808 17:45:32.589888 32497 ts_tablet_manager.cc:1905] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:32.591531 32497 ts_tablet_manager.cc:1918] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:32.591598 32497 log.cc:1196] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/a566086c59c541a8885cb2df4ec15c12
I20260808 17:45:32.591815 32497 ts_tablet_manager.cc:1939] T a566086c59c541a8885cb2df4ec15c12 P 5b8df666afaf41d98048260d34fd6ade: Deleting consensus metadata
I20260808 17:45:32.592144 31645 catalog_manager.cc:4841] TS 5b8df666afaf41d98048260d34fd6ade (127.29.62.195:38157): tablet a566086c59c541a8885cb2df4ec15c12 (table test-table [id=06402a615dd64c7cb8cb8c2ae2ff351a]) successfully deleted
Aug 08 17:45:32 dist-test-slave-tk9z krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:32.601022 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.589840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.29.62.193:43307)
0808 17:45:32.590467 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.590469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.590709 (+ 240us) server_negotiation.cc:407] Connection header received
0808 17:45:32.590815 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.590817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.590828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.590860 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:32.591731 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.592426 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.593511 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.593820 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.593898 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.597452 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:32.597460 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:32.597460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:32.597468 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:32.599586 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:32.599998 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:32.600002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:32.600002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:32.600075 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:32.600310 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:32.600311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:32.600311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:32.600519 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:32.600666 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.600792 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.600925 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599}
W20260808 17:45:32.601269 31968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55078
I20260808 17:45:32.609891 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:32.602219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58502 (local address 127.29.62.254:37141)
0808 17:45:32.602264 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:32.602265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:32.602529 (+ 264us) server_negotiation.cc:407] Connection header received
0808 17:45:32.602564 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:32.602565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:32.602577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:32.602647 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:32.603257 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.604007 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.605027 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:32.605327 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:32.605408 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:32.606900 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:32.606901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:32.606901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:32.606909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:32.608593 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:32.608997 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:32.608998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:32.608998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:32.609069 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:32.609298 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:32.609299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:32.609299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:32.609442 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:32.609563 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:32.609715 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 17:45:32.609800 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 17:45:32.610114 31662 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58502
I20260808 17:45:32.610811 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31782
I20260808 17:45:32.620920 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31986
I20260808 17:45:32.629711 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 32219
I20260808 17:45:32.644587 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31622
2026-08-08T17:45:32Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2694 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:32 dist-test-slave-tk9z krb5kdc[32558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:32 dist-test-slave-tk9z krb5kdc[32558](info): set up 2 sockets
Aug 08 17:45:32 dist-test-slave-tk9z krb5kdc[32558](info): commencing operation
krb5kdc: starting...
W20260808 17:45:33.893229 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:33 dist-test-slave-tk9z krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:34.017096 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:43619
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:43619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:34.100262 32578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:34.100471 32578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:34.100492 32578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:34.101558 32578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:34.101593 32578 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 17:45:34.101605 32578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:34.101619 32578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:34.101631 32578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:34.101643 32578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:34.103154 32578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:43619
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:43619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:34.103354 32578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:34.103533 32578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:34.108283 32586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:34.108299 32585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:34.108299 32588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:34.108654 32578 server_base.cc:1034] running on GCE node
I20260808 17:45:34.108922 32578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:34.109272 32578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:34.110445 32578 hybrid_clock.cc:648] HybridClock initialized: now 1786211134110423 us; error 41 us; skew 500 ppm
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:34.113207 32578 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:34.113708 32578 webserver.cc:460] Webserver started at http://127.29.62.254:46221/ using document root <none> and password file <none>
I20260808 17:45:34.113947 32578 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:34.113993 32578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:34.114167 32578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:34.115336 32578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "d0a52f8d37864b99ad449f0dbc734d1c"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "3ed64b787b7bc8b4a396fef282b81cdf"
server_key_iv: "edb29a1673b0133843f4573121475772"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.115725 32578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "d0a52f8d37864b99ad449f0dbc734d1c"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "3ed64b787b7bc8b4a396fef282b81cdf"
server_key_iv: "edb29a1673b0133843f4573121475772"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.117090 32578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:34.117825 32605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.118005 32578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:34.118112 32578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "d0a52f8d37864b99ad449f0dbc734d1c"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "3ed64b787b7bc8b4a396fef282b81cdf"
server_key_iv: "edb29a1673b0133843f4573121475772"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.118199 32578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:34.130767 32578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:34.132714 32578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:34.132896 32578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:34.137719 32578 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:43619
I20260808 17:45:34.137787 32705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:43619 every 8 connection(s)
I20260808 17:45:34.138125 32578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:34.138839 32706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.141218 32706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: Bootstrap starting.
I20260808 17:45:34.141860 32706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.142161 32706 log.cc:824] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:34.142551 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 32578
I20260808 17:45:34.142650 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:34.142822 29947 external_mini_cluster.cc:1428] Setting key 14fc61525151e29e89bcd4d8a89236f5
I20260808 17:45:34.143211 32706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: No bootstrap required, opened a new log
I20260808 17:45:34.146087 32706 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a52f8d37864b99ad449f0dbc734d1c" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43619 } }
I20260808 17:45:34.146261 32706 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.146296 32706 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a52f8d37864b99ad449f0dbc734d1c, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.146462 32706 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a52f8d37864b99ad449f0dbc734d1c" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43619 } }
I20260808 17:45:34.146548 32706 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:34.146576 32706 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:34.146621 32706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.147284 32706 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a52f8d37864b99ad449f0dbc734d1c" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43619 } }
I20260808 17:45:34.147382 32706 leader_election.cc:302] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0a52f8d37864b99ad449f0dbc734d1c; no voters:
I20260808 17:45:34.147583 32706 leader_election.cc:288] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:34.147634 32713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:34.147792 32713 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [term 1 LEADER]: Becoming Leader. State: Replica: d0a52f8d37864b99ad449f0dbc734d1c, State: Running, Role: LEADER
I20260808 17:45:34.147879 32706 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:34.147912 32713 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a52f8d37864b99ad449f0dbc734d1c" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43619 } }
I20260808 17:45:34.148265 32713 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0a52f8d37864b99ad449f0dbc734d1c. Latest consensus state: current_term: 1 leader_uuid: "d0a52f8d37864b99ad449f0dbc734d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a52f8d37864b99ad449f0dbc734d1c" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43619 } } }
I20260808 17:45:34.148339 32713 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:34.148293 32715 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0a52f8d37864b99ad449f0dbc734d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a52f8d37864b99ad449f0dbc734d1c" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43619 } } }
I20260808 17:45:34.148554 32715 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:34.148631 32721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:34.149260 32721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:34.151176 32721 catalog_manager.cc:1347] Generated new cluster ID: ebf900df16284848a4171d07a8a53a6a
I20260808 17:45:34.151225 32721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:34.155529 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.143396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.29.62.254:43619)
0808 17:45:34.143599 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.143609 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.143654 (+ 45us) server_negotiation.cc:407] Connection header received
0808 17:45:34.144574 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.144584 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.144649 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.144801 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.145488 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.146873 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.147869 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.148478 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.148590 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.151686 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.151690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.151690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.151703 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.153784 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.154180 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.154183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.154183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.154298 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.154560 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.154561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.154561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.154877 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.155033 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.155141 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.155287 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260808 17:45:34.172775 32721 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:34.173571 32721 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:34.184715 32721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: Generated new TSK 0
I20260808 17:45:34.184957 32721 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:34.217792 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:43619
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:34.306280 32745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:34.306541 32745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:34.306569 32745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:34.307653 32745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:34.307691 32745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:34.307713 32745 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 17:45:34.307724 32745 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:45:34.307741 32745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:34.309127 32745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:43619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:34.309315 32745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:34.309490 32745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:34.314399 32765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:34.314361 32763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:34.314424 32761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:34.314694 32745 server_base.cc:1034] running on GCE node
I20260808 17:45:34.314908 32745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:34.315105 32745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:34.316244 32745 hybrid_clock.cc:648] HybridClock initialized: now 1786211134316230 us; error 49 us; skew 500 ppm
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:34.318706 32745 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:34.319152 32745 webserver.cc:460] Webserver started at http://127.29.62.193:35293/ using document root <none> and password file <none>
I20260808 17:45:34.319392 32745 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:34.319439 32745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:34.319618 32745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:34.321009 32745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "a661685d245847f3ada19df6bb56f5eb"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "0a1028b497f343d3d9b5cd94f65efc96"
server_key_iv: "a6de095558de83097b50c7adcefdd545"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.321446 32745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "a661685d245847f3ada19df6bb56f5eb"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "0a1028b497f343d3d9b5cd94f65efc96"
server_key_iv: "a6de095558de83097b50c7adcefdd545"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.323073 32745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 17:45:34.323871 310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.324028 32745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:34.324120 32745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "a661685d245847f3ada19df6bb56f5eb"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "0a1028b497f343d3d9b5cd94f65efc96"
server_key_iv: "a6de095558de83097b50c7adcefdd545"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.324177 32745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:34.335122 32745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:34.336935 32745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:34.337049 32745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:34.337502 32745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:34.337541 32745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.337565 32745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:34.337613 32745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:34.344805 32745 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:42601
I20260808 17:45:34.344892 486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:42601 every 8 connection(s)
I20260808 17:45:34.345222 32745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:34.347725 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.337998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:44083 (local address 127.29.62.254:43619)
0808 17:45:34.338073 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.338076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.338273 (+ 197us) server_negotiation.cc:407] Connection header received
0808 17:45:34.338894 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.338895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.338909 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.338959 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.339647 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.340487 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.342085 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.342455 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.342533 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.344891 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.344893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.344894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.344902 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.346504 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.346873 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.346874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.346875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.346928 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.347155 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.347155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.347156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.347256 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.347306 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.347551 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.347619 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:45:34.347929 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.337972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43619 (local address 127.29.62.193:44083)
0808 17:45:34.338160 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.338200 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.338775 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.339040 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.339044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.339351 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:34.339538 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.339543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.340589 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.340591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.342002 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.342006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.342552 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.342553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.342692 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.343225 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:34.343235 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:34.344775 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:34.346575 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.346582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.346584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.346797 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.346989 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.346990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.346990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.347098 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.347311 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:34.347316 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:34.347424 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.347521 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.347637 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260808 17:45:34.348476 488 heartbeater.cc:347] Connected to a master server at 127.29.62.254:43619
I20260808 17:45:34.348563 488 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:34.348742 488 heartbeater.cc:511] Master 127.29.62.254:43619 requested a full tablet report, sending...
I20260808 17:45:34.349339 32628 ts_manager.cc:194] Registered new tserver with Master: a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601)
I20260808 17:45:34.350059 32628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:44083
I20260808 17:45:34.354307 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 32745
I20260808 17:45:34.354441 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:34.354642 29947 external_mini_cluster.cc:1428] Setting key 203a029ebdd969f9f39fe7bedc74d6bc
I20260808 17:45:34.355222 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.349626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:36187 (local address 127.29.62.254:43619)
0808 17:45:34.350225 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.350226 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.350273 (+ 47us) server_negotiation.cc:407] Connection header received
0808 17:45:34.350294 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.350294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.350305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.350390 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.350788 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.351228 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.351854 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.352035 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.352084 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.352925 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.352926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.352926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.352931 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.354017 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.354467 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.354468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.354469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.354530 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.354729 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.354730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.354730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.354852 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.354952 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.355106 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.355166 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583}
I20260808 17:45:34.355283 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.349564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43619 (local address 127.29.62.193:36187)
0808 17:45:34.349692 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.349701 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.349753 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.350435 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.350436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.350650 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:34.350709 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.350711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.351315 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.351316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.351787 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.351789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.352099 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.352100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.352154 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.352576 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:34.352581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:34.352856 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:34.354087 (+ 1231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.354088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.354088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.354353 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.354585 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.354586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.354587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.354656 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.354944 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:34.354944 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:34.355011 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.355089 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.355188 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:34.379190 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:43619
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:34.462853 502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:34.463053 502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:34.463078 502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:34.464167 502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:34.464202 502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:34.464222 502 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 17:45:34.464234 502 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:45:34.464250 502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:34.465638 502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:43619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:34.465826 502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:34.466006 502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:34.471252 513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:34.471231 515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:34.471292 502 server_base.cc:1034] running on GCE node
W20260808 17:45:34.471292 518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:34.471750 502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:34.472024 502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:34.473166 502 hybrid_clock.cc:648] HybridClock initialized: now 1786211134473156 us; error 28 us; skew 500 ppm
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:34.475782 502 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:34.476227 502 webserver.cc:460] Webserver started at http://127.29.62.194:42365/ using document root <none> and password file <none>
I20260808 17:45:34.476464 502 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:34.476536 502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:34.476691 502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:34.477759 502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "e0c4c2f2c3b54e289ab967f4ca304318"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "8033e02c023a91ef686b2a47aaa0edd3"
server_key_iv: "4c7c21bb7f80cced5d34a20c3bbb70ae"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.478130 502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "e0c4c2f2c3b54e289ab967f4ca304318"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "8033e02c023a91ef686b2a47aaa0edd3"
server_key_iv: "4c7c21bb7f80cced5d34a20c3bbb70ae"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.479540 502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:34.480290 532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.480492 502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:34.480602 502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "e0c4c2f2c3b54e289ab967f4ca304318"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "8033e02c023a91ef686b2a47aaa0edd3"
server_key_iv: "4c7c21bb7f80cced5d34a20c3bbb70ae"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.480679 502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:34.488982 502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:34.491305 502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:34.491465 502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:34.491950 502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:34.492030 502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.492077 502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:34.492101 502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:34.499532 502 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:45753
I20260808 17:45:34.499670 723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:45753 every 8 connection(s)
I20260808 17:45:34.499979 502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:34.502166 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.492543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:38629 (local address 127.29.62.254:43619)
0808 17:45:34.492622 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.492625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.492843 (+ 218us) server_negotiation.cc:407] Connection header received
0808 17:45:34.493587 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.493588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.493599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.493686 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.494656 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.495515 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.496533 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.496748 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.496836 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.499272 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.499274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.499274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.499284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.500911 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.501303 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.501304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.501304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.501362 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.501625 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.501626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.501627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.501755 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.501825 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.502035 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.502102 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:45:34.502334 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.492495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43619 (local address 127.29.62.194:38629)
0808 17:45:34.492697 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.492745 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.493450 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.493785 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.493790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.494184 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:34.494526 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.494534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.495618 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.495619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.496435 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.496437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.496865 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.496866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.497020 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.497546 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:34.497562 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:34.499151 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:34.500986 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.500991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.500993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.501206 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.501428 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.501429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.501429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.501541 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.501826 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:34.501830 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:34.501937 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.502020 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.502140 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
I20260808 17:45:34.502972 725 heartbeater.cc:347] Connected to a master server at 127.29.62.254:43619
I20260808 17:45:34.503082 725 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:34.503263 725 heartbeater.cc:511] Master 127.29.62.254:43619 requested a full tablet report, sending...
I20260808 17:45:34.503743 32628 ts_manager.cc:194] Registered new tserver with Master: e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:34.504238 32628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:38629
I20260808 17:45:34.504554 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 502
I20260808 17:45:34.504664 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:34.504887 29947 external_mini_cluster.cc:1428] Setting key aa19ca062810bbc54241006d808ac7f9
I20260808 17:45:34.510567 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.504066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:34613 (local address 127.29.62.254:43619)
0808 17:45:34.504409 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.504411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.504420 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:34.504440 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.504441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.504451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.504503 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.504901 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.505562 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.506315 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.506666 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.506741 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.507888 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.507890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.507890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.507896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.509298 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.509742 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.509743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.509743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.509810 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.510038 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.510039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.510039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.510177 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.510231 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.510443 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.510507 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322}
I20260808 17:45:34.510677 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.504001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43619 (local address 127.29.62.194:34613)
0808 17:45:34.504129 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.504138 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.504182 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.504530 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.504531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.504728 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:34.504809 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.504812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.505686 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.505687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.506228 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.506230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.506775 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.506776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.506847 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.507309 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:34.507315 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:34.507773 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:34.509389 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.509390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.509391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.509644 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.509891 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.509892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.509893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.509971 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.510249 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:34.510261 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:34.510317 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.510412 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.510535 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:34.531800 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:43619
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:34.615000 736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:34.615195 736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:34.615221 736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:34.616272 736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:34.616305 736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:34.616325 736 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 17:45:34.616338 736 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:45:34.616353 736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:34.617722 736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:43619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:34.617969 736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:34.618222 736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:34.623246 748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:34.623243 746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:34.623243 750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:34.623538 736 server_base.cc:1034] running on GCE node
I20260808 17:45:34.623735 736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:34.623974 736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:34.625139 736 hybrid_clock.cc:648] HybridClock initialized: now 1786211134625127 us; error 28 us; skew 500 ppm
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:34.627758 736 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:34.628201 736 webserver.cc:460] Webserver started at http://127.29.62.195:41791/ using document root <none> and password file <none>
I20260808 17:45:34.628413 736 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:34.628455 736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:34.628546 736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:34.629570 736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "92815f82db924ff290d605f478e4295a"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "f8cf7c9165f04d34226e47c76bf2834e"
server_key_iv: "8ad68da0e1ba706ac4dc96a29c92a498"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.629951 736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "92815f82db924ff290d605f478e4295a"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "f8cf7c9165f04d34226e47c76bf2834e"
server_key_iv: "8ad68da0e1ba706ac4dc96a29c92a498"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.631397 736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:34.632104 760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.632298 736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:34.632385 736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "92815f82db924ff290d605f478e4295a"
format_stamp: "Formatted at 2026-08-08 17:45:34 on dist-test-slave-tk9z"
server_key: "f8cf7c9165f04d34226e47c76bf2834e"
server_key_iv: "8ad68da0e1ba706ac4dc96a29c92a498"
server_key_version: "encryptionkey@0"
I20260808 17:45:34.632440 736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:34.657809 736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:34.660305 736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:34.660429 736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:34.660944 736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:34.660990 736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.661016 736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:34.661029 736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.668025 736 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:37007
I20260808 17:45:34.668089 939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:37007 every 8 connection(s)
I20260808 17:45:34.668469 736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:34.671818 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.661555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:39287 (local address 127.29.62.254:43619)
0808 17:45:34.661627 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.661630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.661912 (+ 282us) server_negotiation.cc:407] Connection header received
0808 17:45:34.662477 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.662479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.662496 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.662584 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.663453 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.664303 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.665892 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.666158 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.666245 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.669124 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.669126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.669126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.669133 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.670682 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.671041 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.671041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.671042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.671094 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.671302 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.671302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.671303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.671408 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.671457 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.671689 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.671752 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:45:34.672029 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.661593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43619 (local address 127.29.62.195:39287)
0808 17:45:34.661771 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.661814 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.662315 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.662660 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.662664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.663033 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:34.663340 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.663348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.664431 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.664432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.665811 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.665815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.666251 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.666252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.666456 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.667107 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:34.667119 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:34.669031 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:34.670753 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.670758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.670760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.670976 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.671152 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.671153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.671153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.671249 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.671462 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:34.671466 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:34.671579 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.671658 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.671777 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260808 17:45:34.672602 940 heartbeater.cc:347] Connected to a master server at 127.29.62.254:43619
I20260808 17:45:34.672693 940 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:34.672885 940 heartbeater.cc:511] Master 127.29.62.254:43619 requested a full tablet report, sending...
I20260808 17:45:34.673259 32628 ts_manager.cc:194] Registered new tserver with Master: 92815f82db924ff290d605f478e4295a (127.29.62.195:37007)
I20260808 17:45:34.673687 32628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:39287
I20260808 17:45:34.677723 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 736
I20260808 17:45:34.677807 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:34.678015 29947 external_mini_cluster.cc:1428] Setting key d2e556bb4fda671e08446ded41d8a964
I20260808 17:45:34.678900 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:45:34.679318 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.673995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:53009 (local address 127.29.62.254:43619)
0808 17:45:34.674045 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.674047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.674055 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:45:34.674081 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.674082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.674106 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.674167 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.674576 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.675062 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.675722 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.675895 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.675947 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.676836 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.676837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.676837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.676842 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.678079 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.678498 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.678499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.678499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.678562 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.678788 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.678789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.678790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.678905 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.678984 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.679148 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.679226 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:45:34.679325 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.673815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43619 (local address 127.29.62.195:53009)
0808 17:45:34.673957 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.673969 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.674033 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.674179 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.674180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.674424 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:34.674498 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.674499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.675147 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.675147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.675651 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.675652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.675959 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.675959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.676016 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.676461 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:34.676467 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:34.676766 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:34.678180 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.678180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.678181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.678431 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.678648 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.678649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.678649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.678718 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.678986 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:34.678987 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:34.679034 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.679121 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.679229 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:34.703645 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.696533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.29.62.254:43619)
0808 17:45:34.696603 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.696605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.696615 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:34.696650 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.696651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.696660 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.696711 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.697284 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.697788 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.698554 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.698738 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.698792 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.700977 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.700978 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.700978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.700985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.702192 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.702646 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.702647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.702648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.702740 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.703017 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.703019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.703019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.703187 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.703298 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.703471 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.703541 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:45:34.705286 32628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:34.706961 32628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:34.713682 964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.709301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42601 (local address 127.0.0.1:40590)
0808 17:45:34.709467 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.709509 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.709578 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.709931 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.709935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.709937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.710005 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.710006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.711454 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.711456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.712398 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.712400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.713362 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.713363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.713444 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.713457 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.713535 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.713586 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260808 17:45:34.713716 970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.709561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.29.62.193:42601)
0808 17:45:34.709760 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.709763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.709772 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:34.709807 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.709809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.709821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.709924 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.710104 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.711310 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.712517 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.713186 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.713258 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.713324 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.713568 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.713616 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
I20260808 17:45:34.714128 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.709303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45753 (local address 127.0.0.1:57730)
0808 17:45:34.710025 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.710036 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.710089 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.710362 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.710362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.710364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.710479 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.710481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.711632 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.711633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.712792 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.712795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.713901 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.713903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.713968 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.713972 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.714015 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.714061 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":2,"thread_start_us":45,"threads_started":1}
I20260808 17:45:34.714511 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.709769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.29.62.195:37007)
0808 17:45:34.709938 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.709943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.710320 (+ 377us) server_negotiation.cc:407] Connection header received
0808 17:45:34.710410 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.710413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.710429 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.710507 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.710602 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.711760 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.713236 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.714109 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.714185 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.714334 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.714360 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.714416 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260808 17:45:34.714505 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.709460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.29.62.194:45753)
0808 17:45:34.709638 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.709642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.710123 (+ 481us) server_negotiation.cc:407] Connection header received
0808 17:45:34.710173 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.710175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.710192 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.710290 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.710565 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.711519 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.712887 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.713626 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.713689 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.714302 (+ 613us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.714341 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.714404 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":77,"threads_started":1}
I20260808 17:45:34.714588 382 tablet_service.cc:1467] Processing CreateTablet for tablet 50a46fc63306476d89391fd4a54c70f2 (DEFAULT_TABLE table=test-table [id=60d1b422de104243bf13f041c9a5af3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:34.714965 382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a46fc63306476d89391fd4a54c70f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.714491 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.709331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37007 (local address 127.0.0.1:36316)
0808 17:45:34.710165 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.710176 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.710216 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.710502 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.710503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.710505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.710541 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.710542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.711863 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.711864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.713149 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.713151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.714202 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.714203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.714281 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.714286 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.714337 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.714421 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":106,"threads_started":1}
I20260808 17:45:34.715261 614 tablet_service.cc:1467] Processing CreateTablet for tablet 50a46fc63306476d89391fd4a54c70f2 (DEFAULT_TABLE table=test-table [id=60d1b422de104243bf13f041c9a5af3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:34.715639 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a46fc63306476d89391fd4a54c70f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.715610 837 tablet_service.cc:1467] Processing CreateTablet for tablet 50a46fc63306476d89391fd4a54c70f2 (DEFAULT_TABLE table=test-table [id=60d1b422de104243bf13f041c9a5af3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:34.715955 837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a46fc63306476d89391fd4a54c70f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.717979 973 tablet_bootstrap.cc:492] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Bootstrap starting.
I20260808 17:45:34.718727 973 tablet_bootstrap.cc:654] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.719050 973 log.cc:824] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:34.719367 977 tablet_bootstrap.cc:492] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Bootstrap starting.
I20260808 17:45:34.719872 973 tablet_bootstrap.cc:492] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: No bootstrap required, opened a new log
I20260808 17:45:34.719954 973 ts_tablet_manager.cc:1397] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:34.720333 977 tablet_bootstrap.cc:654] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.720652 977 log.cc:824] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:34.721103 976 tablet_bootstrap.cc:492] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Bootstrap starting.
I20260808 17:45:34.721731 976 tablet_bootstrap.cc:654] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.722064 976 log.cc:824] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:34.722206 973 raft_consensus.cc:348] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.722397 973 raft_consensus.cc:374] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.722445 973 raft_consensus.cc:729] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a661685d245847f3ada19df6bb56f5eb, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.722576 973 consensus_queue.cc:260] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.722856 973 ts_tablet_manager.cc:1428] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 17:45:34.722918 976 tablet_bootstrap.cc:492] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: No bootstrap required, opened a new log
I20260808 17:45:34.722936 977 tablet_bootstrap.cc:492] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: No bootstrap required, opened a new log
I20260808 17:45:34.722982 488 heartbeater.cc:503] Master 127.29.62.254:43619 was elected leader, sending a full tablet report...
I20260808 17:45:34.722990 976 ts_tablet_manager.cc:1397] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:34.722996 977 ts_tablet_manager.cc:1397] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 17:45:34.724871 976 raft_consensus.cc:348] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.725023 976 raft_consensus.cc:374] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.724958 977 raft_consensus.cc:348] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.725067 976 raft_consensus.cc:729] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c4c2f2c3b54e289ab967f4ca304318, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.725112 977 raft_consensus.cc:374] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.725155 977 raft_consensus.cc:729] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92815f82db924ff290d605f478e4295a, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.725184 976 consensus_queue.cc:260] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.725265 977 consensus_queue.cc:260] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.725455 976 ts_tablet_manager.cc:1428] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:45:34.725526 725 heartbeater.cc:503] Master 127.29.62.254:43619 was elected leader, sending a full tablet report...
I20260808 17:45:34.725565 977 ts_tablet_manager.cc:1428] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:45:34.725654 940 heartbeater.cc:503] Master 127.29.62.254:43619 was elected leader, sending a full tablet report...
W20260808 17:45:34.751248 726 tablet.cc:2369] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:45:34.836894 986 raft_consensus.cc:482] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:34.837086 986 raft_consensus.cc:504] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.837469 986 leader_election.cc:288] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:34.841382 970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.837752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:40919 (local address 127.29.62.193:42601)
0808 17:45:34.837805 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.837807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.837970 (+ 163us) server_negotiation.cc:407] Connection header received
0808 17:45:34.838006 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.838007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.838014 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.838058 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.838219 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.838943 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.840423 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.840984 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.841064 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.841073 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.841273 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.841314 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:45:34.841382 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.837733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:39917 (local address 127.29.62.194:45753)
0808 17:45:34.837779 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.837782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.837817 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:45:34.837855 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.837856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.837863 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.837906 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.838139 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.838927 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.840423 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.841003 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.841065 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.841073 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.841273 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.841313 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:45:34.841383 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.837662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45753 (local address 127.29.62.195:39917)
0808 17:45:34.837711 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.837725 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.837777 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.837940 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.837941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.837943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.838030 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.838032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.839022 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.839023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.840323 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.840327 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.841083 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.841084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.841195 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.841198 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.841251 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.841288 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":106}
I20260808 17:45:34.841393 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.837672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42601 (local address 127.29.62.195:40919)
0808 17:45:34.837866 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.837873 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.837956 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.838081 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.838082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.838083 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.838158 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.838159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.839022 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.839023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.840323 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.840327 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.841083 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.841084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.841173 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.841178 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.841240 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.841288 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":54,"thread_start_us":123,"threads_started":1}
I20260808 17:45:34.841774 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a46fc63306476d89391fd4a54c70f2" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" is_pre_election: true
I20260808 17:45:34.841944 649 raft_consensus.cc:2454] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 0.
I20260808 17:45:34.842177 762 leader_election.cc:302] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92815f82db924ff290d605f478e4295a, e0c4c2f2c3b54e289ab967f4ca304318; no voters:
I20260808 17:45:34.842258 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a46fc63306476d89391fd4a54c70f2" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb" is_pre_election: true
I20260808 17:45:34.842306 986 raft_consensus.cc:2781] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:34.842356 986 raft_consensus.cc:482] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:34.842413 986 raft_consensus.cc:3037] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.842437 415 raft_consensus.cc:2454] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 0.
I20260808 17:45:34.843158 986 raft_consensus.cc:504] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.843307 986 leader_election.cc:288] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 election: Requested vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:34.843466 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a46fc63306476d89391fd4a54c70f2" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb"
I20260808 17:45:34.843456 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a46fc63306476d89391fd4a54c70f2" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318"
I20260808 17:45:34.843541 415 raft_consensus.cc:3037] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.843541 649 raft_consensus.cc:3037] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.844419 649 raft_consensus.cc:2454] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 1.
I20260808 17:45:34.844419 415 raft_consensus.cc:2454] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 1.
I20260808 17:45:34.844617 762 leader_election.cc:302] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92815f82db924ff290d605f478e4295a, e0c4c2f2c3b54e289ab967f4ca304318; no voters:
I20260808 17:45:34.844727 986 raft_consensus.cc:2781] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:34.844885 986 raft_consensus.cc:686] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 1 LEADER]: Becoming Leader. State: Replica: 92815f82db924ff290d605f478e4295a, State: Running, Role: LEADER
I20260808 17:45:34.845001 986 consensus_queue.cc:237] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
W20260808 17:45:34.845985 491 tablet.cc:2369] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:45:34.845919 32628 catalog_manager.cc:5495] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a reported cstate change: term changed from 0 to 1, leader changed from <none> to 92815f82db924ff290d605f478e4295a (127.29.62.195). New cstate: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } health_report { overall_health: HEALTHY } } }
Aug 08 17:45:34 dist-test-slave-tk9z krb5kdc[32558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211134, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:34.886300 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.879189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51990 (local address 127.29.62.254:43619)
0808 17:45:34.879371 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.879373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.879403 (+ 30us) server_negotiation.cc:407] Connection header received
0808 17:45:34.879427 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.879427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.879436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.879488 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:34.879962 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.880444 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.881132 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.881306 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.881354 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.883515 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:34.883516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:34.883517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:34.883523 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:34.885023 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.885427 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.885429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.885429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.885492 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:34.885704 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:34.885705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:34.885705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:34.885814 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:34.885882 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.886054 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.886166 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":151}
I20260808 17:45:34.886408 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.879091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43619 (local address 127.0.0.1:51990)
0808 17:45:34.879264 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.879274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.879365 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.879498 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.879499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.879837 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:34.879888 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.879891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.880524 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.880526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.881059 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.881061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.881369 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.881369 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.881414 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.881898 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:34.881903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:34.883368 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:34.885123 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.885132 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.885134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.885336 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.885560 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:34.885561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:34.885562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:34.885623 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:34.885886 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:34.885894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:34.885938 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.886029 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.886230 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":92,"thread_start_us":80,"threads_started":1}
I20260808 17:45:34.888389 32628 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:51990:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 17:45:34.888582 32628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:34.890173 614 tablet_service.cc:1467] Processing CreateTablet for tablet d30f9409f1b54eaf907d6a468b976a43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=271e65296c3c47d499511236555d2640]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:45:34.890146 382 tablet_service.cc:1467] Processing CreateTablet for tablet d30f9409f1b54eaf907d6a468b976a43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=271e65296c3c47d499511236555d2640]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:45:34.890432 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30f9409f1b54eaf907d6a468b976a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.890429 382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30f9409f1b54eaf907d6a468b976a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.890482 837 tablet_service.cc:1467] Processing CreateTablet for tablet d30f9409f1b54eaf907d6a468b976a43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=271e65296c3c47d499511236555d2640]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:45:34.890641 837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30f9409f1b54eaf907d6a468b976a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.891703 976 tablet_bootstrap.cc:492] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: Bootstrap starting.
I20260808 17:45:34.892112 973 tablet_bootstrap.cc:492] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: Bootstrap starting.
I20260808 17:45:34.892166 977 tablet_bootstrap.cc:492] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: Bootstrap starting.
I20260808 17:45:34.892463 976 tablet_bootstrap.cc:654] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.892575 973 tablet_bootstrap.cc:654] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.892588 977 tablet_bootstrap.cc:654] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.893275 976 tablet_bootstrap.cc:492] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: No bootstrap required, opened a new log
I20260808 17:45:34.893332 977 tablet_bootstrap.cc:492] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: No bootstrap required, opened a new log
I20260808 17:45:34.893329 976 ts_tablet_manager.cc:1397] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:34.893373 977 ts_tablet_manager.cc:1397] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:34.893462 973 tablet_bootstrap.cc:492] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: No bootstrap required, opened a new log
I20260808 17:45:34.893499 973 ts_tablet_manager.cc:1397] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 17:45:34.893497 976 raft_consensus.cc:348] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.893512 977 raft_consensus.cc:348] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.893584 977 raft_consensus.cc:374] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.893584 976 raft_consensus.cc:374] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.893606 976 raft_consensus.cc:729] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c4c2f2c3b54e289ab967f4ca304318, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.893606 977 raft_consensus.cc:729] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92815f82db924ff290d605f478e4295a, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.893661 977 consensus_queue.cc:260] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.893661 976 consensus_queue.cc:260] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.893625 973 raft_consensus.cc:348] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.893715 973 raft_consensus.cc:374] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.893738 973 raft_consensus.cc:729] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a661685d245847f3ada19df6bb56f5eb, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.893750 976 ts_tablet_manager.cc:1428] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:34.893761 977 ts_tablet_manager.cc:1428] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.893784 973 consensus_queue.cc:260] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.893886 973 ts_tablet_manager.cc:1428] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.893903 985 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.893977 985 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:34.893953 986 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.894016 986 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:34.894161 984 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.894245 984 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:34.908934 986 raft_consensus.cc:482] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:34.909040 986 raft_consensus.cc:504] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.909219 986 leader_election.cc:288] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:34.909392 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30f9409f1b54eaf907d6a468b976a43" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb" is_pre_election: true
I20260808 17:45:34.909440 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30f9409f1b54eaf907d6a468b976a43" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" is_pre_election: true
I20260808 17:45:34.909514 415 raft_consensus.cc:2454] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 0.
I20260808 17:45:34.909538 649 raft_consensus.cc:2454] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 0.
I20260808 17:45:34.909731 762 leader_election.cc:302] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92815f82db924ff290d605f478e4295a, e0c4c2f2c3b54e289ab967f4ca304318; no voters:
I20260808 17:45:34.909878 986 raft_consensus.cc:2781] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:34.909919 986 raft_consensus.cc:482] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:34.909951 986 raft_consensus.cc:3037] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.911010 986 raft_consensus.cc:504] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.911187 986 leader_election.cc:288] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 election: Requested vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:34.911466 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30f9409f1b54eaf907d6a468b976a43" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb"
I20260808 17:45:34.911468 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30f9409f1b54eaf907d6a468b976a43" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318"
I20260808 17:45:34.911546 415 raft_consensus.cc:3037] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.911545 649 raft_consensus.cc:3037] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.912328 415 raft_consensus.cc:2454] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 1.
I20260808 17:45:34.912416 649 raft_consensus.cc:2454] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 1.
I20260808 17:45:34.912518 761 leader_election.cc:302] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92815f82db924ff290d605f478e4295a, a661685d245847f3ada19df6bb56f5eb; no voters:
I20260808 17:45:34.912627 986 raft_consensus.cc:2781] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:34.912724 986 raft_consensus.cc:686] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [term 1 LEADER]: Becoming Leader. State: Replica: 92815f82db924ff290d605f478e4295a, State: Running, Role: LEADER
I20260808 17:45:34.912788 986 consensus_queue.cc:237] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.913143 988 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: TxnStatusTablet state changed. Reason: New leader 92815f82db924ff290d605f478e4295a. Latest consensus state: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.913234 988 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:45:34.913512 1005 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 17:45:34.913676 1005 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 17:45:34.913764 32624 catalog_manager.cc:5495] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a reported cstate change: term changed from 0 to 1, leader changed from <none> to 92815f82db924ff290d605f478e4295a (127.29.62.195). New cstate: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } health_report { overall_health: HEALTHY } } }
W20260808 17:45:34.919072 941 tablet.cc:2369] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:45:34.924633 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.921787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37007 (local address 127.0.0.1:36320)
0808 17:45:34.921934 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.921946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.922012 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.922178 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.922179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.922180 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:45:34.922227 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.922229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.923132 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.923132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.923682 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.923683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.924175 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.924176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.924223 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.924238 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:45:34.924508 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:45:34.924514 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.924542 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.924577 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 17:45:34.924643 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.921849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.29.62.195:37007)
0808 17:45:34.921900 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.921902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.922016 (+ 114us) server_negotiation.cc:407] Connection header received
0808 17:45:34.922074 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.922076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.922097 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.922179 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:34.922275 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.923054 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.923769 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.924084 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.924170 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.924327 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:34.924437 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:34.924524 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.924547 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.924570 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:45:34.926569 649 raft_consensus.cc:1261] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Refusing update from remote peer 92815f82db924ff290d605f478e4295a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:34.926613 415 raft_consensus.cc:1261] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Refusing update from remote peer 92815f82db924ff290d605f478e4295a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:34.926915 986 consensus_queue.cc:1035] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:34.927059 988 consensus_queue.cc:1035] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:34.928351 984 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: TxnStatusTablet state changed. Reason: New leader 92815f82db924ff290d605f478e4295a. Latest consensus state: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.928447 984 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:34.928433 985 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: TxnStatusTablet state changed. Reason: New leader 92815f82db924ff290d605f478e4295a. Latest consensus state: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.928565 985 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:34.929811 988 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.929908 988 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:45:34.929955 984 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.930042 984 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P a661685d245847f3ada19df6bb56f5eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:34.930127 1010 mvcc.cc:205] Tried to move back new op lower bound from 7316320808656547840 to 7316320808603189248. Current Snapshot: MvccSnapshot[applied={T|T < 7316320808656547840}]
I20260808 17:45:34.930768 1009 mvcc.cc:205] Tried to move back new op lower bound from 7316320808656547840 to 7316320808603189248. Current Snapshot: MvccSnapshot[applied={T|T < 7316320808656547840 or (T in {7316320808656547840})}]
I20260808 17:45:34.934758 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.932159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36324 (local address 127.29.62.195:37007)
0808 17:45:34.932195 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.932197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.932211 (+ 14us) server_negotiation.cc:407] Connection header received
0808 17:45:34.932275 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.932276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.932286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.932357 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:34.932477 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.933207 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.933925 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.934232 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.934305 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.934474 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:34.934512 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:34.934571 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.934648 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.934671 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:45:34.935717 986 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.935829 986 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P 92815f82db924ff290d605f478e4295a: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:45:34.936285 985 tablet_replica.cc:439] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } } }
I20260808 17:45:34.936394 985 tablet_replica.cc:442] T d30f9409f1b54eaf907d6a468b976a43 P e0c4c2f2c3b54e289ab967f4ca304318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:45:34.940243 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.937937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:42009 (local address 127.29.62.195:37007)
0808 17:45:34.937982 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.937984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.937993 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:34.938051 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.938052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.938061 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.938114 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:34.938249 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.938768 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.939511 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.939708 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.939782 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.939984 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:34.940014 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:34.940081 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.940136 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.940160 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":24}
I20260808 17:45:34.940243 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.937865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37007 (local address 127.29.62.195:42009)
0808 17:45:34.937941 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.937953 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.938000 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.938117 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.938118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.938119 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:45:34.938182 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.938184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.938848 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.938849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.939404 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.939405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.939783 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.939783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.939853 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.939875 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:45:34.940083 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:45:34.940086 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.940134 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.940176 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":5}
I20260808 17:45:34.943095 415 raft_consensus.cc:1261] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Refusing update from remote peer 92815f82db924ff290d605f478e4295a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:34.943104 649 raft_consensus.cc:1261] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Refusing update from remote peer 92815f82db924ff290d605f478e4295a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:34.943323 1021 consensus_queue.cc:1035] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:34.943452 986 consensus_queue.cc:1035] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:34.946468 1011 mvcc.cc:205] Tried to move back new op lower bound from 7316320808725250048 to 7316320808325877760. Current Snapshot: MvccSnapshot[applied={T|T < 7316320808725250048}]
I20260808 17:45:34.958500 32624 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:34.958623 32624 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:34.959416 32624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: Sending DeleteTablet for 3 replicas of tablet 50a46fc63306476d89391fd4a54c70f2
I20260808 17:45:34.959722 382 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a46fc63306476d89391fd4a54c70f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:34 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:40590
I20260808 17:45:34.959770 614 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a46fc63306476d89391fd4a54c70f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:34 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:57730
I20260808 17:45:34.960155 837 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a46fc63306476d89391fd4a54c70f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:34 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:36316
I20260808 17:45:34.960206 32624 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:34.960357 32624 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:34.960667 1031 tablet_replica.cc:330] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: stopping tablet replica
I20260808 17:45:34.960789 1031 raft_consensus.cc:2229] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:34.960863 1031 raft_consensus.cc:2258] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:34.961093 1029 tablet_replica.cc:330] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: stopping tablet replica
I20260808 17:45:34.961213 1029 raft_consensus.cc:2229] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:34.961268 1029 raft_consensus.cc:2258] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:34.961603 382 tablet_service.cc:1467] Processing CreateTablet for tablet 5b821295f86c4fa0b5a273589dcb7683 (DEFAULT_TABLE table=test-table [id=2c748ba0cef348198f2404795553c896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:34.961727 1029 ts_tablet_manager.cc:1905] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:34.961755 382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b821295f86c4fa0b5a273589dcb7683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.961738 614 tablet_service.cc:1467] Processing CreateTablet for tablet 5b821295f86c4fa0b5a273589dcb7683 (DEFAULT_TABLE table=test-table [id=2c748ba0cef348198f2404795553c896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:34.961849 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b821295f86c4fa0b5a273589dcb7683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.963717 976 tablet_bootstrap.cc:492] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: Bootstrap starting.
I20260808 17:45:34.963743 973 tablet_bootstrap.cc:492] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: Bootstrap starting.
I20260808 17:45:34.963800 1029 ts_tablet_manager.cc:1918] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:34.963879 1029 log.cc:1196] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/50a46fc63306476d89391fd4a54c70f2
I20260808 17:45:34.964170 976 tablet_bootstrap.cc:654] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.964170 973 tablet_bootstrap.cc:654] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.964186 1029 ts_tablet_manager.cc:1939] T 50a46fc63306476d89391fd4a54c70f2 P a661685d245847f3ada19df6bb56f5eb: Deleting consensus metadata
I20260808 17:45:34.964648 837 tablet_service.cc:1467] Processing CreateTablet for tablet 5b821295f86c4fa0b5a273589dcb7683 (DEFAULT_TABLE table=test-table [id=2c748ba0cef348198f2404795553c896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:34.964797 837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b821295f86c4fa0b5a273589dcb7683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:34.964958 32608 catalog_manager.cc:4841] TS a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601): tablet 50a46fc63306476d89391fd4a54c70f2 (table test-table [id=60d1b422de104243bf13f041c9a5af3c]) successfully deleted
I20260808 17:45:34.965302 973 tablet_bootstrap.cc:492] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: No bootstrap required, opened a new log
I20260808 17:45:34.965348 973 ts_tablet_manager.cc:1397] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260808 17:45:34.965461 976 tablet_bootstrap.cc:492] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: No bootstrap required, opened a new log
I20260808 17:45:34.965504 976 ts_tablet_manager.cc:1397] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:34.965484 973 raft_consensus.cc:348] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.965551 973 raft_consensus.cc:374] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.965571 973 raft_consensus.cc:729] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a661685d245847f3ada19df6bb56f5eb, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.965620 973 consensus_queue.cc:260] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.965641 976 raft_consensus.cc:348] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.965701 976 raft_consensus.cc:374] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.965720 976 raft_consensus.cc:729] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c4c2f2c3b54e289ab967f4ca304318, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.965721 973 ts_tablet_manager.cc:1428] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:34.965770 976 consensus_queue.cc:260] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.965860 976 ts_tablet_manager.cc:1428] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:34.966310 977 tablet_bootstrap.cc:492] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: Bootstrap starting.
I20260808 17:45:34.966756 977 tablet_bootstrap.cc:654] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:34.967499 1031 ts_tablet_manager.cc:1905] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:34.967638 977 tablet_bootstrap.cc:492] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: No bootstrap required, opened a new log
I20260808 17:45:34.967715 977 ts_tablet_manager.cc:1397] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:34.967876 977 raft_consensus.cc:348] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.967952 977 raft_consensus.cc:374] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:34.967986 977 raft_consensus.cc:729] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92815f82db924ff290d605f478e4295a, State: Initialized, Role: FOLLOWER
I20260808 17:45:34.968037 977 consensus_queue.cc:260] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.968128 977 ts_tablet_manager.cc:1428] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:34.969045 1031 ts_tablet_manager.cc:1918] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:34.969125 1031 log.cc:1196] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/50a46fc63306476d89391fd4a54c70f2
I20260808 17:45:34.969388 1031 ts_tablet_manager.cc:1939] T 50a46fc63306476d89391fd4a54c70f2 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting consensus metadata
I20260808 17:45:34.969792 32609 catalog_manager.cc:4841] TS e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753): tablet 50a46fc63306476d89391fd4a54c70f2 (table test-table [id=60d1b422de104243bf13f041c9a5af3c]) successfully deleted
I20260808 17:45:34.970479 1034 tablet_replica.cc:330] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: stopping tablet replica
I20260808 17:45:34.970599 1034 raft_consensus.cc:2229] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:34.970721 1034 raft_consensus.cc:2258] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:34.971101 1034 ts_tablet_manager.cc:1905] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:34.972666 1034 ts_tablet_manager.cc:1918] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:34.972741 1034 log.cc:1196] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/50a46fc63306476d89391fd4a54c70f2
I20260808 17:45:34.973061 1034 ts_tablet_manager.cc:1939] T 50a46fc63306476d89391fd4a54c70f2 P 92815f82db924ff290d605f478e4295a: Deleting consensus metadata
I20260808 17:45:34.973883 32610 catalog_manager.cc:4841] TS 92815f82db924ff290d605f478e4295a (127.29.62.195:37007): tablet 50a46fc63306476d89391fd4a54c70f2 (table test-table [id=60d1b422de104243bf13f041c9a5af3c]) successfully deleted
I20260808 17:45:34.979592 984 raft_consensus.cc:482] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:34.979671 984 raft_consensus.cc:504] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.980015 984 leader_election.cc:288] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753), 92815f82db924ff290d605f478e4295a (127.29.62.195:37007)
I20260808 17:45:34.980988 985 raft_consensus.cc:482] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:34.981057 985 raft_consensus.cc:504] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.981344 985 leader_election.cc:288] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), 92815f82db924ff290d605f478e4295a (127.29.62.195:37007)
I20260808 17:45:34.984082 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.980331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:46965 (local address 127.29.62.194:45753)
0808 17:45:34.980373 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.980375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.980588 (+ 213us) server_negotiation.cc:407] Connection header received
0808 17:45:34.980623 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.980624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.980634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.980709 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.980853 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.981576 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.983184 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.983725 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.983766 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.983775 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.983979 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.984017 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:45:34.984292 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.980177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45753 (local address 127.29.62.193:46965)
0808 17:45:34.980522 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.980532 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.980579 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.980710 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.980711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.980712 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.980775 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.980776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.981666 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.981667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.983084 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.983087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.983830 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.983831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.983904 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.983907 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.984160 (+ 253us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.984210 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":70,"threads_started":1}
I20260808 17:45:34.984611 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b821295f86c4fa0b5a273589dcb7683" candidate_uuid: "a661685d245847f3ada19df6bb56f5eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" is_pre_election: true
I20260808 17:45:34.984696 649 raft_consensus.cc:2454] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a661685d245847f3ada19df6bb56f5eb in term 0.
I20260808 17:45:34.984882 312 leader_election.cc:302] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a661685d245847f3ada19df6bb56f5eb, e0c4c2f2c3b54e289ab967f4ca304318; no voters:
I20260808 17:45:34.985008 984 raft_consensus.cc:2781] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:34.985038 984 raft_consensus.cc:482] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:34.985082 984 raft_consensus.cc:3037] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.985322 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.980162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37007 (local address 127.29.62.193:52129)
0808 17:45:34.980363 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.980375 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.980440 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.980762 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.980764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.980766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.980849 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.980851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.982521 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.982522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.983590 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.983593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.984875 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.984876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.984947 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.984952 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.984999 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.985040 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":57,"threads_started":1}
I20260808 17:45:34.985399 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.981473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37007 (local address 127.29.62.194:55955)
0808 17:45:34.981516 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.981525 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.981557 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.981929 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.981931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.981932 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.981991 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.981993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.983102 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.983103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.984318 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.984321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.985184 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.985185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.985262 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.985265 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.985295 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.985330 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":3}
I20260808 17:45:34.985863 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.980275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:52129 (local address 127.29.62.195:37007)
0808 17:45:34.980369 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.980371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.980436 (+ 65us) server_negotiation.cc:407] Connection header received
0808 17:45:34.980517 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.980518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.980529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.980584 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.980910 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.982429 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.983702 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.984679 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.985717 (+ 1038us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.985731 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.985753 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.985788 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 17:45:34.986014 984 raft_consensus.cc:504] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.986150 984 leader_election.cc:288] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [CANDIDATE]: Term 1 election: Requested vote from peers e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753), 92815f82db924ff290d605f478e4295a (127.29.62.195:37007)
I20260808 17:45:34.986191 875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b821295f86c4fa0b5a273589dcb7683" candidate_uuid: "a661685d245847f3ada19df6bb56f5eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92815f82db924ff290d605f478e4295a" is_pre_election: true
I20260808 17:45:34.986320 875 raft_consensus.cc:2454] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a661685d245847f3ada19df6bb56f5eb in term 0.
I20260808 17:45:34.986315 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b821295f86c4fa0b5a273589dcb7683" candidate_uuid: "a661685d245847f3ada19df6bb56f5eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318"
I20260808 17:45:34.986440 649 raft_consensus.cc:3037] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.986465 874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b821295f86c4fa0b5a273589dcb7683" candidate_uuid: "a661685d245847f3ada19df6bb56f5eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92815f82db924ff290d605f478e4295a"
I20260808 17:45:34.986562 874 raft_consensus.cc:3037] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:34.986763 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.981690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:55955 (local address 127.29.62.195:37007)
0808 17:45:34.981785 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.981787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.981794 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:45:34.981824 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.981825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.981833 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.982051 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.982080 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.982980 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.984479 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.985084 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.986611 (+ 1527us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.986621 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.986640 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.986687 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":55,"threads_started":1}
I20260808 17:45:34.986914 875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b821295f86c4fa0b5a273589dcb7683" candidate_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92815f82db924ff290d605f478e4295a" is_pre_election: true
I20260808 17:45:34.987161 649 raft_consensus.cc:2454] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a661685d245847f3ada19df6bb56f5eb in term 1.
I20260808 17:45:34.987304 312 leader_election.cc:302] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a661685d245847f3ada19df6bb56f5eb, e0c4c2f2c3b54e289ab967f4ca304318; no voters:
I20260808 17:45:34.987385 874 raft_consensus.cc:2454] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a661685d245847f3ada19df6bb56f5eb in term 1.
I20260808 17:45:34.987416 984 raft_consensus.cc:2781] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:34.987565 984 raft_consensus.cc:686] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 1 LEADER]: Becoming Leader. State: Replica: a661685d245847f3ada19df6bb56f5eb, State: Running, Role: LEADER
I20260808 17:45:34.987629 984 consensus_queue.cc:237] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:34.988519 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.981523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42601 (local address 127.29.62.194:35803)
0808 17:45:34.985467 (+ 3944us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:34.985477 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:34.985519 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:34.985737 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:34.985738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:34.985739 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:34.985767 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.985768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.986564 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.986566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.987592 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:34.987595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.988294 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.988295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:34.988340 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.988343 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:34.988416 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 17:45:34.988450 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3933,"thread_start_us":67,"threads_started":1}
I20260808 17:45:34.988530 970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.981670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:35803 (local address 127.29.62.193:42601)
0808 17:45:34.981706 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.981708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.985572 (+ 3864us) server_negotiation.cc:407] Connection header received
0808 17:45:34.985596 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.985598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.985606 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.985649 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:34.985848 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.986489 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.987664 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.988210 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.988286 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.988293 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:34.988424 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 17:45:34.988465 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:45:34.988674 32629 catalog_manager.cc:5495] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb reported cstate change: term changed from 0 to 1, leader changed from <none> to a661685d245847f3ada19df6bb56f5eb (127.29.62.193). New cstate: current_term: 1 leader_uuid: "a661685d245847f3ada19df6bb56f5eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } health_report { overall_health: UNKNOWN } } }
I20260808 17:45:34.989085 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b821295f86c4fa0b5a273589dcb7683" candidate_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb" is_pre_election: true
I20260808 17:45:34.989341 533 leader_election.cc:302] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0c4c2f2c3b54e289ab967f4ca304318; no voters: 92815f82db924ff290d605f478e4295a, a661685d245847f3ada19df6bb56f5eb
I20260808 17:45:34.989498 985 raft_consensus.cc:2726] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 17:45:35.000264 970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:34.997907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.29.62.193:42601)
0808 17:45:34.997949 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:34.997951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:34.997961 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:34.998036 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:34.998037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:34.998057 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:34.998142 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:34.998245 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.998777 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.999498 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:34.999703 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:34.999777 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:34.999933 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:35.000039 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:35.000125 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:35.000165 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 17:45:35.000186 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:45:35.005545 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:35.002974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:42841 (local address 127.29.62.195:37007)
0808 17:45:35.003071 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:35.003073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:35.003139 (+ 66us) server_negotiation.cc:407] Connection header received
0808 17:45:35.003189 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:35.003190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:35.003201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:35.003266 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:35.003394 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:35.004072 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:35.004783 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:35.005095 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:35.005170 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:35.005316 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:35.005349 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:35.005402 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:35.005447 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 17:45:35.005471 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 17:45:35.005565 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:35.002911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37007 (local address 127.29.62.193:42841)
0808 17:45:35.003085 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:35.003095 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:35.003146 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:35.003274 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:35.003275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:35.003276 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:45:35.003335 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:35.003336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:35.004156 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:35.004157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:35.004695 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:35.004697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:35.005177 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:35.005178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:35.005225 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:35.005238 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:45:35.005408 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:45:35.005410 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:45:35.005444 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 17:45:35.005476 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260808 17:45:35.007685 649 raft_consensus.cc:1261] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Refusing update from remote peer a661685d245847f3ada19df6bb56f5eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:35.007743 874 raft_consensus.cc:1261] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Refusing update from remote peer a661685d245847f3ada19df6bb56f5eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:35.007929 984 consensus_queue.cc:1035] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:35.008066 1047 consensus_queue.cc:1035] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:35.016269 970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:35.013813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:43051 (local address 127.29.62.193:42601)
0808 17:45:35.013858 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:35.013860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:35.013869 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:35.013955 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:35.013957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:35.013967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:35.014016 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:35.014180 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:35.014762 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:35.015460 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:35.015722 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:35.015791 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:35.015962 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:35.015993 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:35.016065 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:35.016180 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 17:45:35.016198 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:45:35.016280 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:35.013779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42601 (local address 127.29.62.195:43051)
0808 17:45:35.013833 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:35.013843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:35.013879 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:35.014059 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:35.014060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:35.014061 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:45:35.014105 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:35.014106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:35.014846 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:35.014847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:35.015374 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:35.015376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:35.015811 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:35.015812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:35.015879 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:35.015885 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:45:35.016113 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:45:35.016115 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:45:35.016175 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:45:35.016217 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:45:35.016778 1010 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 17:45:36.537228 32629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:36.537335 32629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:36.538228 32629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: Sending DeleteTablet for 3 replicas of tablet 5b821295f86c4fa0b5a273589dcb7683
I20260808 17:45:36.538537 382 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b821295f86c4fa0b5a273589dcb7683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:36 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:40590
I20260808 17:45:36.538573 614 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b821295f86c4fa0b5a273589dcb7683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:36 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:57730
I20260808 17:45:36.538900 1097 tablet_replica.cc:330] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: stopping tablet replica
I20260808 17:45:36.539011 1097 raft_consensus.cc:2229] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:36.539026 836 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b821295f86c4fa0b5a273589dcb7683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:36 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:36316
I20260808 17:45:36.539188 1100 tablet_replica.cc:330] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: stopping tablet replica
I20260808 17:45:36.539237 1100 raft_consensus.cc:2229] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:36.539287 1100 raft_consensus.cc:2258] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:36.539423 1097 raft_consensus.cc:2258] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:36.539701 1097 ts_tablet_manager.cc:1905] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:36.541445 1097 ts_tablet_manager.cc:1918] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:45:36.541517 1097 log.cc:1196] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/5b821295f86c4fa0b5a273589dcb7683
I20260808 17:45:36.541822 1100 ts_tablet_manager.cc:1905] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:36.541846 1097 ts_tablet_manager.cc:1939] T 5b821295f86c4fa0b5a273589dcb7683 P a661685d245847f3ada19df6bb56f5eb: Deleting consensus metadata
I20260808 17:45:36.542109 1099 tablet_replica.cc:330] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: stopping tablet replica
I20260808 17:45:36.542201 1099 raft_consensus.cc:2229] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:36.542191 32608 catalog_manager.cc:4841] TS a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601): tablet 5b821295f86c4fa0b5a273589dcb7683 (table test-table [id=2c748ba0cef348198f2404795553c896]) successfully deleted
I20260808 17:45:36.542241 1099 raft_consensus.cc:2258] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:36.542541 1099 ts_tablet_manager.cc:1905] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:36.543428 1100 ts_tablet_manager.cc:1918] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:45:36.543495 1100 log.cc:1196] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/5b821295f86c4fa0b5a273589dcb7683
I20260808 17:45:36.543711 1100 ts_tablet_manager.cc:1939] T 5b821295f86c4fa0b5a273589dcb7683 P 92815f82db924ff290d605f478e4295a: Deleting consensus metadata
I20260808 17:45:36.544051 32610 catalog_manager.cc:4841] TS 92815f82db924ff290d605f478e4295a (127.29.62.195:37007): tablet 5b821295f86c4fa0b5a273589dcb7683 (table test-table [id=2c748ba0cef348198f2404795553c896]) successfully deleted
I20260808 17:45:36.544099 1099 ts_tablet_manager.cc:1918] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:45:36.544143 1099 log.cc:1196] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/5b821295f86c4fa0b5a273589dcb7683
I20260808 17:45:36.544381 1099 ts_tablet_manager.cc:1939] T 5b821295f86c4fa0b5a273589dcb7683 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting consensus metadata
I20260808 17:45:36.544864 32609 catalog_manager.cc:4841] TS e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753): tablet 5b821295f86c4fa0b5a273589dcb7683 (table test-table [id=2c748ba0cef348198f2404795553c896]) successfully deleted
Aug 08 17:45:36 dist-test-slave-tk9z krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211136, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:45:36 dist-test-slave-tk9z krb5kdc[32558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211136, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:36.571151 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:36.558634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52006 (local address 127.29.62.254:43619)
0808 17:45:36.558810 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:36.558812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:36.558822 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:36.558870 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:36.558871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:36.558881 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:36.558993 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:36.559649 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:36.560167 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:36.560978 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:36.561193 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:36.561292 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:36.568106 (+ 6814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:36.568111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:36.568115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:36.568125 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:36.569719 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:36.570159 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:36.570160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:36.570161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:36.570225 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:36.570563 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:36.570565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:36.570565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:36.570706 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:36.570824 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:36.570932 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 17:45:36.571031 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":115,"threads_started":1}
I20260808 17:45:36.572962 32629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:36.573119 32629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:36.574661 836 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8e46d8186749649054cb2b33a58798 (DEFAULT_TABLE table=test-table [id=b9abc92a401b4925a573ec16566ca258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:36.574661 614 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8e46d8186749649054cb2b33a58798 (DEFAULT_TABLE table=test-table [id=b9abc92a401b4925a573ec16566ca258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:36.574672 382 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8e46d8186749649054cb2b33a58798 (DEFAULT_TABLE table=test-table [id=b9abc92a401b4925a573ec16566ca258]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:36.574816 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8e46d8186749649054cb2b33a58798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:36.574817 836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8e46d8186749649054cb2b33a58798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:36.574836 382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8e46d8186749649054cb2b33a58798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:36.575955 1111 tablet_bootstrap.cc:492] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: Bootstrap starting.
I20260808 17:45:36.576293 1112 tablet_bootstrap.cc:492] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: Bootstrap starting.
I20260808 17:45:36.576397 1111 tablet_bootstrap.cc:654] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:36.576509 1113 tablet_bootstrap.cc:492] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: Bootstrap starting.
I20260808 17:45:36.576844 1112 tablet_bootstrap.cc:654] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:36.576902 1113 tablet_bootstrap.cc:654] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:36.577163 1111 tablet_bootstrap.cc:492] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: No bootstrap required, opened a new log
I20260808 17:45:36.577255 1111 ts_tablet_manager.cc:1397] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:36.577397 1111 raft_consensus.cc:348] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.577476 1112 tablet_bootstrap.cc:492] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: No bootstrap required, opened a new log
I20260808 17:45:36.577491 1111 raft_consensus.cc:374] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:36.577514 1111 raft_consensus.cc:729] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a661685d245847f3ada19df6bb56f5eb, State: Initialized, Role: FOLLOWER
I20260808 17:45:36.577518 1112 ts_tablet_manager.cc:1397] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:36.577587 1111 consensus_queue.cc:260] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.577620 1112 raft_consensus.cc:348] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.577682 1112 raft_consensus.cc:374] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:36.577706 1112 raft_consensus.cc:729] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c4c2f2c3b54e289ab967f4ca304318, State: Initialized, Role: FOLLOWER
I20260808 17:45:36.577747 1112 consensus_queue.cc:260] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.577801 1111 ts_tablet_manager.cc:1428] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:36.577930 1112 ts_tablet_manager.cc:1428] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:36.578290 1113 tablet_bootstrap.cc:492] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: No bootstrap required, opened a new log
I20260808 17:45:36.578393 1113 ts_tablet_manager.cc:1397] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:36.578547 1113 raft_consensus.cc:348] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.578620 1113 raft_consensus.cc:374] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:36.578634 1113 raft_consensus.cc:729] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92815f82db924ff290d605f478e4295a, State: Initialized, Role: FOLLOWER
I20260808 17:45:36.578714 1113 consensus_queue.cc:260] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.578811 1113 ts_tablet_manager.cc:1428] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:36.710343 1077 raft_consensus.cc:482] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:36.710490 1077 raft_consensus.cc:504] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.710695 1077 leader_election.cc:288] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:36.710953 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8e46d8186749649054cb2b33a58798" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb" is_pre_election: true
I20260808 17:45:36.710963 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8e46d8186749649054cb2b33a58798" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" is_pre_election: true
I20260808 17:45:36.711076 415 raft_consensus.cc:2454] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 0.
I20260808 17:45:36.711072 649 raft_consensus.cc:2454] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 0.
I20260808 17:45:36.711287 761 leader_election.cc:302] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92815f82db924ff290d605f478e4295a, a661685d245847f3ada19df6bb56f5eb; no voters:
I20260808 17:45:36.711434 1077 raft_consensus.cc:2781] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:36.711535 1077 raft_consensus.cc:482] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:36.711567 1077 raft_consensus.cc:3037] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:36.712329 1077 raft_consensus.cc:504] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.712509 1077 leader_election.cc:288] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 election: Requested vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:36.712677 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8e46d8186749649054cb2b33a58798" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb"
I20260808 17:45:36.712764 415 raft_consensus.cc:3037] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:36.712749 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8e46d8186749649054cb2b33a58798" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318"
I20260808 17:45:36.712853 649 raft_consensus.cc:3037] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:36.713389 415 raft_consensus.cc:2454] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 1.
I20260808 17:45:36.713630 761 leader_election.cc:302] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92815f82db924ff290d605f478e4295a, a661685d245847f3ada19df6bb56f5eb; no voters:
I20260808 17:45:36.713658 649 raft_consensus.cc:2454] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 1.
I20260808 17:45:36.713757 1077 raft_consensus.cc:2781] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:36.713819 1077 raft_consensus.cc:686] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 1 LEADER]: Becoming Leader. State: Replica: 92815f82db924ff290d605f478e4295a, State: Running, Role: LEADER
I20260808 17:45:36.713881 1077 consensus_queue.cc:237] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.714643 32628 catalog_manager.cc:5495] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a reported cstate change: term changed from 0 to 1, leader changed from <none> to 92815f82db924ff290d605f478e4295a (127.29.62.195). New cstate: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } health_report { overall_health: HEALTHY } } }
I20260808 17:45:36.726285 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:36.723765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36340 (local address 127.29.62.195:37007)
0808 17:45:36.723925 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:36.723927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:36.723933 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:45:36.723952 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:36.723953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:36.723961 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:36.724048 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:36.724235 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:36.724727 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:36.725519 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:36.725736 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:36.725817 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:36.725997 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:36.726040 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:36.726109 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:36.726171 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 17:45:36.726193 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260808 17:45:36.728518 649 raft_consensus.cc:1261] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Refusing update from remote peer 92815f82db924ff290d605f478e4295a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:36.728679 415 raft_consensus.cc:1261] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Refusing update from remote peer 92815f82db924ff290d605f478e4295a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:36.728734 1077 consensus_queue.cc:1035] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:36.728966 1077 consensus_queue.cc:1035] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:36.739523 32624 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:36.739617 32624 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:36.740243 32624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: Sending DeleteTablet for 3 replicas of tablet 7c8e46d8186749649054cb2b33a58798
I20260808 17:45:36.740505 614 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c8e46d8186749649054cb2b33a58798 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:36 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:57730
I20260808 17:45:36.740530 837 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c8e46d8186749649054cb2b33a58798 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:36 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:36316
I20260808 17:45:36.740648 1099 tablet_replica.cc:330] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: stopping tablet replica
I20260808 17:45:36.740669 382 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c8e46d8186749649054cb2b33a58798 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:36 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:40590
I20260808 17:45:36.740718 1099 raft_consensus.cc:2229] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:36.740754 1097 tablet_replica.cc:330] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: stopping tablet replica
I20260808 17:45:36.740772 1099 raft_consensus.cc:2258] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:36.740809 1097 raft_consensus.cc:2229] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:36.740844 1097 raft_consensus.cc:2258] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:36.740840 1100 tablet_replica.cc:330] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: stopping tablet replica
I20260808 17:45:36.740902 1100 raft_consensus.cc:2229] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:36.740833 32624 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 17:45:36.740964 1100 raft_consensus.cc:2258] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 17:45:36.740993 32624 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:36.741205 1097 ts_tablet_manager.cc:1905] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:36.741354 1099 ts_tablet_manager.cc:1905] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:36.741397 1100 ts_tablet_manager.cc:1905] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:36.742117 614 tablet_service.cc:1467] Processing CreateTablet for tablet d85b71cc22ed4c3b8841b9b0d2122e2f (DEFAULT_TABLE table=test-table [id=a060277906a5416ea749a411c00774fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:36.742311 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d85b71cc22ed4c3b8841b9b0d2122e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:36.742897 1099 ts_tablet_manager.cc:1918] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:36.742897 1097 ts_tablet_manager.cc:1918] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:36.742951 1097 log.cc:1196] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/7c8e46d8186749649054cb2b33a58798
I20260808 17:45:36.742951 1099 log.cc:1196] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/7c8e46d8186749649054cb2b33a58798
I20260808 17:45:36.743063 1100 ts_tablet_manager.cc:1918] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:45:36.743115 1100 log.cc:1196] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/7c8e46d8186749649054cb2b33a58798
I20260808 17:45:36.743186 1097 ts_tablet_manager.cc:1939] T 7c8e46d8186749649054cb2b33a58798 P a661685d245847f3ada19df6bb56f5eb: Deleting consensus metadata
I20260808 17:45:36.743186 1099 ts_tablet_manager.cc:1939] T 7c8e46d8186749649054cb2b33a58798 P e0c4c2f2c3b54e289ab967f4ca304318: Deleting consensus metadata
I20260808 17:45:36.743358 1100 ts_tablet_manager.cc:1939] T 7c8e46d8186749649054cb2b33a58798 P 92815f82db924ff290d605f478e4295a: Deleting consensus metadata
I20260808 17:45:36.743633 1112 tablet_bootstrap.cc:492] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: Bootstrap starting.
I20260808 17:45:36.743727 32609 catalog_manager.cc:4841] TS e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753): tablet 7c8e46d8186749649054cb2b33a58798 (table test-table [id=b9abc92a401b4925a573ec16566ca258]) successfully deleted
I20260808 17:45:36.743785 32610 catalog_manager.cc:4841] TS 92815f82db924ff290d605f478e4295a (127.29.62.195:37007): tablet 7c8e46d8186749649054cb2b33a58798 (table test-table [id=b9abc92a401b4925a573ec16566ca258]) successfully deleted
I20260808 17:45:36.744086 1112 tablet_bootstrap.cc:654] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:36.744125 382 tablet_service.cc:1467] Processing CreateTablet for tablet d85b71cc22ed4c3b8841b9b0d2122e2f (DEFAULT_TABLE table=test-table [id=a060277906a5416ea749a411c00774fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:36.744228 382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d85b71cc22ed4c3b8841b9b0d2122e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:36.744432 837 tablet_service.cc:1467] Processing CreateTablet for tablet d85b71cc22ed4c3b8841b9b0d2122e2f (DEFAULT_TABLE table=test-table [id=a060277906a5416ea749a411c00774fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:36.744535 837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d85b71cc22ed4c3b8841b9b0d2122e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:36.745491 1111 tablet_bootstrap.cc:492] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb: Bootstrap starting.
I20260808 17:45:36.745512 32608 catalog_manager.cc:4841] TS a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601): tablet 7c8e46d8186749649054cb2b33a58798 (table test-table [id=b9abc92a401b4925a573ec16566ca258]) successfully deleted
I20260808 17:45:36.745747 1113 tablet_bootstrap.cc:492] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: Bootstrap starting.
I20260808 17:45:36.745854 1111 tablet_bootstrap.cc:654] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:36.745872 1112 tablet_bootstrap.cc:492] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: No bootstrap required, opened a new log
I20260808 17:45:36.745921 1112 ts_tablet_manager.cc:1397] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:36.746068 1113 tablet_bootstrap.cc:654] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:36.746037 1112 raft_consensus.cc:348] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.746109 1112 raft_consensus.cc:374] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:36.746131 1112 raft_consensus.cc:729] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c4c2f2c3b54e289ab967f4ca304318, State: Initialized, Role: FOLLOWER
I20260808 17:45:36.746174 1112 consensus_queue.cc:260] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.746274 1112 ts_tablet_manager.cc:1428] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:36.746493 1111 tablet_bootstrap.cc:492] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb: No bootstrap required, opened a new log
I20260808 17:45:36.746547 1111 ts_tablet_manager.cc:1397] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:36.746647 1111 raft_consensus.cc:348] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.746733 1111 raft_consensus.cc:374] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:36.746766 1111 raft_consensus.cc:729] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a661685d245847f3ada19df6bb56f5eb, State: Initialized, Role: FOLLOWER
I20260808 17:45:36.746811 1111 consensus_queue.cc:260] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.746905 1111 ts_tablet_manager.cc:1428] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:36.747284 1113 tablet_bootstrap.cc:492] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: No bootstrap required, opened a new log
I20260808 17:45:36.747359 1113 ts_tablet_manager.cc:1397] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:36.747468 1113 raft_consensus.cc:348] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.747561 1113 raft_consensus.cc:374] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:36.747593 1113 raft_consensus.cc:729] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92815f82db924ff290d605f478e4295a, State: Initialized, Role: FOLLOWER
I20260808 17:45:36.747638 1113 consensus_queue.cc:260] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.747781 1113 ts_tablet_manager.cc:1428] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:36.968322 1021 raft_consensus.cc:482] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:36.968484 1021 raft_consensus.cc:504] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.968732 1021 leader_election.cc:288] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:36.969038 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d85b71cc22ed4c3b8841b9b0d2122e2f" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb" is_pre_election: true
I20260808 17:45:36.969038 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d85b71cc22ed4c3b8841b9b0d2122e2f" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" is_pre_election: true
I20260808 17:45:36.969164 415 raft_consensus.cc:2454] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 0.
I20260808 17:45:36.969164 649 raft_consensus.cc:2454] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 0.
I20260808 17:45:36.969365 761 leader_election.cc:302] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92815f82db924ff290d605f478e4295a, a661685d245847f3ada19df6bb56f5eb; no voters:
I20260808 17:45:36.969533 1021 raft_consensus.cc:2781] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:36.969583 1021 raft_consensus.cc:482] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:36.969609 1021 raft_consensus.cc:3037] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:36.970517 1021 raft_consensus.cc:504] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.970672 1021 leader_election.cc:288] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 election: Requested vote from peers a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601), e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753)
I20260808 17:45:36.970831 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d85b71cc22ed4c3b8841b9b0d2122e2f" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0c4c2f2c3b54e289ab967f4ca304318"
I20260808 17:45:36.970839 415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d85b71cc22ed4c3b8841b9b0d2122e2f" candidate_uuid: "92815f82db924ff290d605f478e4295a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a661685d245847f3ada19df6bb56f5eb"
I20260808 17:45:36.970920 649 raft_consensus.cc:3037] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:36.970942 415 raft_consensus.cc:3037] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:36.971652 415 raft_consensus.cc:2454] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 1.
I20260808 17:45:36.971729 649 raft_consensus.cc:2454] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92815f82db924ff290d605f478e4295a in term 1.
I20260808 17:45:36.971840 761 leader_election.cc:302] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92815f82db924ff290d605f478e4295a, a661685d245847f3ada19df6bb56f5eb; no voters:
I20260808 17:45:36.971953 1021 raft_consensus.cc:2781] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:36.972018 1021 raft_consensus.cc:686] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 1 LEADER]: Becoming Leader. State: Replica: 92815f82db924ff290d605f478e4295a, State: Running, Role: LEADER
I20260808 17:45:36.972070 1021 consensus_queue.cc:237] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } }
I20260808 17:45:36.972762 32624 catalog_manager.cc:5495] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a reported cstate change: term changed from 0 to 1, leader changed from <none> to 92815f82db924ff290d605f478e4295a (127.29.62.195). New cstate: current_term: 1 leader_uuid: "92815f82db924ff290d605f478e4295a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92815f82db924ff290d605f478e4295a" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37007 } health_report { overall_health: HEALTHY } } }
I20260808 17:45:37.033188 415 raft_consensus.cc:1261] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Refusing update from remote peer 92815f82db924ff290d605f478e4295a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:37.033206 649 raft_consensus.cc:1261] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Refusing update from remote peer 92815f82db924ff290d605f478e4295a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:37.033452 1068 consensus_queue.cc:1035] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a661685d245847f3ada19df6bb56f5eb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:37.033596 1021 consensus_queue.cc:1035] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0c4c2f2c3b54e289ab967f4ca304318" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 17:45:37.038467 1063 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 17:45:38.556382 32629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:38.556495 32629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:38.557442 32629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0a52f8d37864b99ad449f0dbc734d1c: Sending DeleteTablet for 3 replicas of tablet d85b71cc22ed4c3b8841b9b0d2122e2f
I20260808 17:45:38.557761 382 tablet_service.cc:1514] Processing DeleteTablet for tablet d85b71cc22ed4c3b8841b9b0d2122e2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:38 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:40590
I20260808 17:45:38.557758 614 tablet_service.cc:1514] Processing DeleteTablet for tablet d85b71cc22ed4c3b8841b9b0d2122e2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:38 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:57730
I20260808 17:45:38.557848 837 tablet_service.cc:1514] Processing DeleteTablet for tablet d85b71cc22ed4c3b8841b9b0d2122e2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:38 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:36316
I20260808 17:45:38.558023 1164 tablet_replica.cc:330] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: stopping tablet replica
I20260808 17:45:38.558035 1163 tablet_replica.cc:330] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb: stopping tablet replica
I20260808 17:45:38.558095 1163 raft_consensus.cc:2229] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:38.558188 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 32745
I20260808 17:45:38.558182 1163 raft_consensus.cc:2258] T d85b71cc22ed4c3b8841b9b0d2122e2f P a661685d245847f3ada19df6bb56f5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:38.571332 1162 tablet_replica.cc:330] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: stopping tablet replica
I20260808 17:45:38.571327 1164 raft_consensus.cc:2229] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:38.571422 1162 raft_consensus.cc:2229] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:38.571458 1164 raft_consensus.cc:2258] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:38.571463 1162 raft_consensus.cc:2258] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:38.572361 1164 ts_tablet_manager.cc:1905] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:38.572361 1162 ts_tablet_manager.cc:1905] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:38.574232 1162 ts_tablet_manager.cc:1918] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:45:38.574234 1164 ts_tablet_manager.cc:1918] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 17:45:38.574299 1164 log.cc:1196] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/d85b71cc22ed4c3b8841b9b0d2122e2f
I20260808 17:45:38.574299 1162 log.cc:1196] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/d85b71cc22ed4c3b8841b9b0d2122e2f
I20260808 17:45:38.574568 1162 ts_tablet_manager.cc:1939] T d85b71cc22ed4c3b8841b9b0d2122e2f P e0c4c2f2c3b54e289ab967f4ca304318: Deleting consensus metadata
I20260808 17:45:38.574584 1164 ts_tablet_manager.cc:1939] T d85b71cc22ed4c3b8841b9b0d2122e2f P 92815f82db924ff290d605f478e4295a: Deleting consensus metadata
I20260808 17:45:38.574993 32609 catalog_manager.cc:4841] TS e0c4c2f2c3b54e289ab967f4ca304318 (127.29.62.194:45753): tablet d85b71cc22ed4c3b8841b9b0d2122e2f (table test-table [id=a060277906a5416ea749a411c00774fd]) successfully deleted
I20260808 17:45:38.575059 32610 catalog_manager.cc:4841] TS 92815f82db924ff290d605f478e4295a (127.29.62.195:37007): tablet d85b71cc22ed4c3b8841b9b0d2122e2f (table test-table [id=a060277906a5416ea749a411c00774fd]) successfully deleted
W20260808 17:45:38.578231 32608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:42601 (error 108)
I20260808 17:45:38.578502 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 502
I20260808 17:45:38.587977 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:38.581883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42601 (local address 127.0.0.1:40600)
0808 17:45:38.587885 (+ 6002us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:38.587936 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:42601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5960,"thread_start_us":75,"threads_started":1}
I20260808 17:45:38.588117 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 736
W20260808 17:45:38.588183 32608 catalog_manager.cc:4584] TS a661685d245847f3ada19df6bb56f5eb (127.29.62.193:42601): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d85b71cc22ed4c3b8841b9b0d2122e2f: Network error: Client connection negotiation failed: client connection to 127.29.62.193:42601: connect: Connection refused (error 111)
I20260808 17:45:38.602701 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 32578
2026-08-08T17:45:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5955 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:38 dist-test-slave-tk9z krb5kdc[1170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:38 dist-test-slave-tk9z krb5kdc[1170](info): set up 2 sockets
Aug 08 17:45:38 dist-test-slave-tk9z krb5kdc[1170](info): commencing operation
krb5kdc: starting...
W20260808 17:45:39.854667 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:39 dist-test-slave-tk9z krb5kdc[1170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:39.981170 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:38301
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:38301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:40.066340 1189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:40.066690 1189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:40.066732 1189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:40.067948 1189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:40.068037 1189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:40.068066 1189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:40.068099 1189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:40.068125 1189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:40.069634 1189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:38301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:38301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:40.069921 1189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:40.070232 1189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:40.078943 1202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:40.079087 1198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:40.078954 1196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:40.079507 1189 server_base.cc:1034] running on GCE node
I20260808 17:45:40.079788 1189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:40.080129 1189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:40.081336 1189 hybrid_clock.cc:648] HybridClock initialized: now 1786211140081302 us; error 42 us; skew 500 ppm
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:40.084892 1189 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:40.085561 1189 webserver.cc:460] Webserver started at http://127.29.62.254:39207/ using document root <none> and password file <none>
I20260808 17:45:40.085832 1189 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:40.085898 1189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:40.086071 1189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:40.087894 1189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "a46591e3ac1945ad818c13cf0fbe39a6"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "da73f8b65f2addcb730300e4d772f63e"
server_key_iv: "f19aace811390fefd8f0046178044333"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.088375 1189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "a46591e3ac1945ad818c13cf0fbe39a6"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "da73f8b65f2addcb730300e4d772f63e"
server_key_iv: "f19aace811390fefd8f0046178044333"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.090359 1189 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 17:45:40.091702 1212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.093109 1189 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 17:45:40.093231 1189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "a46591e3ac1945ad818c13cf0fbe39a6"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "da73f8b65f2addcb730300e4d772f63e"
server_key_iv: "f19aace811390fefd8f0046178044333"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.093328 1189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:40.115516 1189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:40.118098 1189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:40.118280 1189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:40.122318 1189 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:38301
I20260808 17:45:40.122429 1290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:38301 every 8 connection(s)
I20260808 17:45:40.122820 1189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:40.124049 1291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:40.127000 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 1189
I20260808 17:45:40.127100 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:40.127305 29947 external_mini_cluster.cc:1428] Setting key f059d29c7500f7e159292acefd58dc14
I20260808 17:45:40.127995 1291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6: Bootstrap starting.
I20260808 17:45:40.128820 1291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:40.129211 1291 log.cc:824] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:40.129983 1291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6: No bootstrap required, opened a new log
I20260808 17:45:40.131888 1291 raft_consensus.cc:348] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46591e3ac1945ad818c13cf0fbe39a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38301 } }
I20260808 17:45:40.132072 1291 raft_consensus.cc:374] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:40.132138 1291 raft_consensus.cc:729] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a46591e3ac1945ad818c13cf0fbe39a6, State: Initialized, Role: FOLLOWER
I20260808 17:45:40.132302 1291 consensus_queue.cc:260] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46591e3ac1945ad818c13cf0fbe39a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38301 } }
I20260808 17:45:40.132398 1291 raft_consensus.cc:388] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:40.132452 1291 raft_consensus.cc:482] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:40.132503 1291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:40.133302 1291 raft_consensus.cc:504] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46591e3ac1945ad818c13cf0fbe39a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38301 } }
I20260808 17:45:40.133430 1291 leader_election.cc:302] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a46591e3ac1945ad818c13cf0fbe39a6; no voters:
I20260808 17:45:40.133610 1291 leader_election.cc:288] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:40.133648 1299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:40.133816 1299 raft_consensus.cc:686] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [term 1 LEADER]: Becoming Leader. State: Replica: a46591e3ac1945ad818c13cf0fbe39a6, State: Running, Role: LEADER
I20260808 17:45:40.133901 1291 sys_catalog.cc:564] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:40.133996 1299 consensus_queue.cc:237] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46591e3ac1945ad818c13cf0fbe39a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38301 } }
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:40.134332 1299 sys_catalog.cc:455] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a46591e3ac1945ad818c13cf0fbe39a6. Latest consensus state: current_term: 1 leader_uuid: "a46591e3ac1945ad818c13cf0fbe39a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46591e3ac1945ad818c13cf0fbe39a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38301 } } }
I20260808 17:45:40.134343 1301 sys_catalog.cc:455] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a46591e3ac1945ad818c13cf0fbe39a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46591e3ac1945ad818c13cf0fbe39a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38301 } } }
I20260808 17:45:40.134482 1299 sys_catalog.cc:458] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:40.134485 1301 sys_catalog.cc:458] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:40.135293 1304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:40.136559 1304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:40.138549 1304 catalog_manager.cc:1347] Generated new cluster ID: 1b6a7e00bf354a1fa4d1e41b8ba460de
I20260808 17:45:40.138651 1304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:40.138985 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:40.127841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38136 (local address 127.29.62.254:38301)
0808 17:45:40.128032 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:40.128040 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:40.128095 (+ 55us) server_negotiation.cc:407] Connection header received
0808 17:45:40.128532 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:40.128539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:40.128586 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:40.128749 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:40.129479 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.130541 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.131311 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.131730 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.131855 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:40.135035 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:40.135039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:40.135040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:40.135056 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:40.137234 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:40.137614 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:40.137617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:40.137618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:40.137737 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:40.137982 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:40.137983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:40.137983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:40.138289 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:40.138474 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:40.138627 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 17:45:40.138781 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260808 17:45:40.151083 1304 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:40.151722 1304 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:40.158697 1304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a46591e3ac1945ad818c13cf0fbe39a6: Generated new TSK 0
I20260808 17:45:40.158923 1304 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:40.192813 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38301
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:40.275553 1327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:40.275758 1327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:40.275782 1327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:40.277062 1327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:40.277109 1327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:40.277136 1327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:40.278672 1327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:40.278856 1327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:40.279042 1327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:40.283969 1336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:40.283980 1340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:40.283980 1334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:40.284335 1327 server_base.cc:1034] running on GCE node
I20260808 17:45:40.284541 1327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:40.284734 1327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:40.285871 1327 hybrid_clock.cc:648] HybridClock initialized: now 1786211140285861 us; error 26 us; skew 500 ppm
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:40.288517 1327 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:40.288955 1327 webserver.cc:460] Webserver started at http://127.29.62.193:41741/ using document root <none> and password file <none>
I20260808 17:45:40.289186 1327 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:40.289230 1327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:40.289381 1327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:40.290562 1327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "b9a540099cdf4ed8ad7c2f06747d0513"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "45e4c2d2d3c0157955d3d175dd5fdec5"
server_key_iv: "c23870e782ed34fc86f097e153c3a8ef"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.290967 1327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "b9a540099cdf4ed8ad7c2f06747d0513"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "45e4c2d2d3c0157955d3d175dd5fdec5"
server_key_iv: "c23870e782ed34fc86f097e153c3a8ef"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.292308 1327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:45:40.293028 1348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.293239 1327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:45:40.293332 1327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "b9a540099cdf4ed8ad7c2f06747d0513"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "45e4c2d2d3c0157955d3d175dd5fdec5"
server_key_iv: "c23870e782ed34fc86f097e153c3a8ef"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.293388 1327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:40.307611 1327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:40.309362 1327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:40.309484 1327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:40.309659 1327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:40.310002 1327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:40.310038 1327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.310060 1327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:40.310115 1327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.315958 1327 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:34785
I20260808 17:45:40.316040 1536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:34785 every 8 connection(s)
I20260808 17:45:40.316403 1327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:40.318048 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 1327
I20260808 17:45:40.318189 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:40.318394 29947 external_mini_cluster.cc:1428] Setting key 6fcee8f8f9ea3f537ff9fb5ff775f4ef
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:40.327548 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:40.317201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:35009 (local address 127.29.62.254:38301)
0808 17:45:40.317248 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:40.317251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:40.317439 (+ 188us) server_negotiation.cc:407] Connection header received
0808 17:45:40.318073 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:40.318075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:40.318087 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:40.318185 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:40.319557 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.320046 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.321424 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.321645 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.321739 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:40.324331 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:40.324332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:40.324333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:40.324342 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:40.326039 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:40.326476 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:40.326478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:40.326478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:40.326550 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:40.326823 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:40.326823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:40.326824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:40.326974 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:40.327047 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:40.327317 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 17:45:40.327420 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:45:40.327600 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:40.317143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38301 (local address 127.29.62.193:35009)
0808 17:45:40.317301 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:40.317349 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:40.317930 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:40.318297 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:40.318302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:40.319162 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:40.319459 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:40.319467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.320146 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.320148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:40.321318 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:40.321321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.321751 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.321752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:40.321884 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:40.322468 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:40.322478 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:40.324227 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:40.326129 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:40.326136 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:40.326137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:40.326357 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:40.326629 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:40.326630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:40.326631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:40.326737 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:40.327046 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:40.327050 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:40.327165 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 17:45:40.327272 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 17:45:40.327423 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260808 17:45:40.328179 1538 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38301
I20260808 17:45:40.328289 1538 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
I20260808 17:45:40.328516 1538 heartbeater.cc:511] Master 127.29.62.254:38301 requested a full tablet report, sending...
I20260808 17:45:40.329056 1237 ts_manager.cc:194] Registered new tserver with Master: b9a540099cdf4ed8ad7c2f06747d0513 (127.29.62.193:34785)
I20260808 17:45:40.329708 1237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:35009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:40.345422 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38301
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:40.429183 1548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:40.429401 1548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:40.429425 1548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:40.430725 1548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:40.430770 1548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:40.430796 1548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:40.432189 1548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:40.432368 1548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:40.432549 1548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:40.437788 1555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:40.437788 1560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:40.437855 1556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:40.438213 1548 server_base.cc:1034] running on GCE node
I20260808 17:45:40.438489 1548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:40.438740 1548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:40.439929 1548 hybrid_clock.cc:648] HybridClock initialized: now 1786211140439915 us; error 29 us; skew 500 ppm
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:40.442631 1548 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:40.443077 1548 webserver.cc:460] Webserver started at http://127.29.62.194:38551/ using document root <none> and password file <none>
I20260808 17:45:40.443316 1548 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:40.443362 1548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:40.443531 1548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:40.444628 1548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "fed500f5d49f46ebad2902a9755b2142"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "3e10ea87ad2bda5530dc51a02a5d7e82"
server_key_iv: "15cc02066d8c0b7bafe941f7e905faed"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.445012 1548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "fed500f5d49f46ebad2902a9755b2142"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "3e10ea87ad2bda5530dc51a02a5d7e82"
server_key_iv: "15cc02066d8c0b7bafe941f7e905faed"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.446339 1548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 17:45:40.447158 1571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.447355 1548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:40.447458 1548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "fed500f5d49f46ebad2902a9755b2142"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "3e10ea87ad2bda5530dc51a02a5d7e82"
server_key_iv: "15cc02066d8c0b7bafe941f7e905faed"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.447540 1548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:40.460225 1548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:40.462867 1548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:40.463071 1548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:40.463335 1548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:40.463693 1548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:40.463754 1548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.463792 1548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:40.463819 1548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.470033 1548 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:44175
I20260808 17:45:40.470086 1746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:44175 every 8 connection(s)
I20260808 17:45:40.470541 1548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:40.470887 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 1548
I20260808 17:45:40.471004 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:40.471172 29947 external_mini_cluster.cc:1428] Setting key 143ac0ad8701f07f1af67b8a007754a8
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
I20260808 17:45:40.481451 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:40.471362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:52375 (local address 127.29.62.254:38301)
0808 17:45:40.471481 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:40.471483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:40.471603 (+ 120us) server_negotiation.cc:407] Connection header received
0808 17:45:40.472309 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:40.472310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:40.472320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:40.472360 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:40.473744 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.474239 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.475611 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.475799 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.475889 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:40.478544 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:40.478550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:40.478550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:40.478557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:40.479955 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:40.480406 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:40.480407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:40.480407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:40.480485 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:40.480763 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:40.480764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:40.480764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:40.480887 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:40.480965 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:40.481272 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 17:45:40.481357 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 17:45:40.481634 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:40.471255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38301 (local address 127.29.62.194:52375)
0808 17:45:40.471477 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:40.471531 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:40.472195 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:40.472440 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:40.472444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:40.473321 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:40.473625 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:40.473633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.474346 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.474347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:40.475527 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:40.475530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.475902 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.475903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:40.476052 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:40.476700 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:40.476711 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:40.478432 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:40.480063 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:40.480071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:40.480073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:40.480321 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:40.480567 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:40.480568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:40.480568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:40.480682 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:40.480968 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:40.480983 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:40.481123 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 17:45:40.481235 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 17:45:40.481398 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":116,"threads_started":1}
I20260808 17:45:40.482180 1748 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38301
I20260808 17:45:40.482301 1748 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:40.482527 1748 heartbeater.cc:511] Master 127.29.62.254:38301 requested a full tablet report, sending...
I20260808 17:45:40.482971 1237 ts_manager.cc:194] Registered new tserver with Master: fed500f5d49f46ebad2902a9755b2142 (127.29.62.194:44175)
I20260808 17:45:40.483314 1237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:52375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:40.501646 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38301
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:40.586679 1757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:40.586889 1757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:40.586912 1757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:40.588017 1757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:40.588055 1757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:40.588080 1757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:40.589442 1757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:40.589627 1757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:40.589795 1757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:40.594921 1769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:40.594928 1773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:40.594933 1763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:40.595319 1757 server_base.cc:1034] running on GCE node
I20260808 17:45:40.595532 1757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:40.595729 1757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:40.596877 1757 hybrid_clock.cc:648] HybridClock initialized: now 1786211140596862 us; error 28 us; skew 500 ppm
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:40.599478 1757 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:40.599954 1757 webserver.cc:460] Webserver started at http://127.29.62.195:33385/ using document root <none> and password file <none>
I20260808 17:45:40.600179 1757 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:40.600247 1757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:40.600404 1757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:40.601418 1757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "4ec5671f0ea841ae9587d3b28ee071eb"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "25a06703c2bbb8c2e934e079ec4643de"
server_key_iv: "f6ea21ef827dca3a3a16f5338072c37c"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.601819 1757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "4ec5671f0ea841ae9587d3b28ee071eb"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "25a06703c2bbb8c2e934e079ec4643de"
server_key_iv: "f6ea21ef827dca3a3a16f5338072c37c"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.603251 1757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:40.604013 1785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.604192 1757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:45:40.604303 1757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "4ec5671f0ea841ae9587d3b28ee071eb"
format_stamp: "Formatted at 2026-08-08 17:45:40 on dist-test-slave-tk9z"
server_key: "25a06703c2bbb8c2e934e079ec4643de"
server_key_iv: "f6ea21ef827dca3a3a16f5338072c37c"
server_key_version: "encryptionkey@0"
I20260808 17:45:40.604398 1757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:40.618192 1757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:40.620509 1757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:40.620679 1757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:40.620904 1757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:40.621227 1757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:40.621284 1757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.621325 1757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:40.621359 1757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:40.627362 1757 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:37553
I20260808 17:45:40.627415 1953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:37553 every 8 connection(s)
I20260808 17:45:40.627735 1757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211140, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:40.637925 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 1757
I20260808 17:45:40.638047 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:40.638191 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:40.628486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:43931 (local address 127.29.62.254:38301)
0808 17:45:40.628531 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:40.628533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:40.628718 (+ 185us) server_negotiation.cc:407] Connection header received
0808 17:45:40.629269 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:40.629271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:40.629285 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:40.629335 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:40.630336 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.630876 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.631890 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.632103 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.632193 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:40.634254 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:40.634256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:40.634256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:40.634263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:40.636682 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:40.637111 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:40.637112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:40.637112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:40.637173 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:40.637447 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:40.637448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:40.637448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:40.637576 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:40.637652 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:40.638038 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 17:45:40.638126 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:45:40.638253 29947 external_mini_cluster.cc:1428] Setting key 0f8a4d29e89192e8c31eca53c66c69f4
I20260808 17:45:40.638420 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:40.628444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38301 (local address 127.29.62.195:43931)
0808 17:45:40.628586 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:40.628631 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:40.629147 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:40.629402 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:40.629407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:40.630003 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:40.630187 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:40.630191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.631000 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.631001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:40.631787 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:40.631789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:40.632207 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:40.632208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:40.632345 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:40.632761 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:40.632769 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:40.634157 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:40.636789 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:40.636797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:40.636799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:40.637024 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:40.637251 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:40.637252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:40.637252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:40.637369 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:40.637656 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:40.637663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:40.637843 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 17:45:40.637981 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 17:45:40.638162 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260808 17:45:40.639027 1954 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38301
I20260808 17:45:40.639156 1954 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:40.639328 1954 heartbeater.cc:511] Master 127.29.62.254:38301 requested a full tablet report, sending...
I20260808 17:45:40.639653 1237 ts_manager.cc:194] Registered new tserver with Master: 4ec5671f0ea841ae9587d3b28ee071eb (127.29.62.195:37553)
I20260808 17:45:40.640038 1237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:43931
I20260808 17:45:40.650174 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 17:45:40.664320 1294 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:40.663261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.29.62.254:38301)
0808 17:45:40.663352 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:40.663354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:40.663460 (+ 106us) server_negotiation.cc:407] Connection header received
0808 17:45:40.663550 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:40.663551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:40.663563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:40.663641 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:40.664222 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38152: BlockingRecv error: recv got EOF from 127.0.0.1:38152 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 17:45:40.665146 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 1327
I20260808 17:45:40.675702 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 1548
I20260808 17:45:40.699719 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 1757
I20260808 17:45:40.711921 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 1189
2026-08-08T17:45:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2107 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1971](info): set up 2 sockets
Aug 08 17:45:40 dist-test-slave-tk9z krb5kdc[1971](info): commencing operation
krb5kdc: starting...
W20260808 17:45:41.951176 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:42 dist-test-slave-tk9z krb5kdc[1971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:42.079327 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:42881
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:42881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:42.163901 1989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:42.164115 1989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:42.164137 1989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:42.165349 1989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:42.165395 1989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:42.165408 1989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:42.165421 1989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:42.165433 1989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:42.167038 1989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:42881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:42881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:42.167309 1989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:42.167552 1989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:42.172565 1994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:42.172565 1997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:42.172626 1995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:42.172694 1989 server_base.cc:1034] running on GCE node
I20260808 17:45:42.173071 1989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:42.173421 1989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:42.174566 1989 hybrid_clock.cc:648] HybridClock initialized: now 1786211142174555 us; error 27 us; skew 500 ppm
Aug 08 17:45:42 dist-test-slave-tk9z krb5kdc[1971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211142, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:42.182354 1989 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:42.182935 1989 webserver.cc:460] Webserver started at http://127.29.62.254:36631/ using document root <none> and password file <none>
I20260808 17:45:42.183193 1989 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:42.183266 1989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:42.183420 1989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:42.184553 1989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "70da431546bb4303bc9314b2d827dda2"
format_stamp: "Formatted at 2026-08-08 17:45:42 on dist-test-slave-tk9z"
server_key: "afb15a6b44e086399a7ef236e58bb04b"
server_key_iv: "e0bd83483839c7155353a17aeb7eaa41"
server_key_version: "encryptionkey@0"
I20260808 17:45:42.184955 1989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "70da431546bb4303bc9314b2d827dda2"
format_stamp: "Formatted at 2026-08-08 17:45:42 on dist-test-slave-tk9z"
server_key: "afb15a6b44e086399a7ef236e58bb04b"
server_key_iv: "e0bd83483839c7155353a17aeb7eaa41"
server_key_version: "encryptionkey@0"
I20260808 17:45:42.186305 1989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 17:45:42.187136 2007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:42.187318 1989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:42.187435 1989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "70da431546bb4303bc9314b2d827dda2"
format_stamp: "Formatted at 2026-08-08 17:45:42 on dist-test-slave-tk9z"
server_key: "afb15a6b44e086399a7ef236e58bb04b"
server_key_iv: "e0bd83483839c7155353a17aeb7eaa41"
server_key_version: "encryptionkey@0"
I20260808 17:45:42.187529 1989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:42.201195 1989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:42.203778 1989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:42.203960 1989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:42.208590 1989 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:42881
I20260808 17:45:42.208628 2087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:42881 every 8 connection(s)
I20260808 17:45:42.209204 1989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:42.209939 2089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:42.212293 2089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2: Bootstrap starting.
I20260808 17:45:42.212852 2089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:42.213107 2089 log.cc:824] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:42.213774 2089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2: No bootstrap required, opened a new log
I20260808 17:45:42.215129 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 1989
I20260808 17:45:42.215247 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:42.215448 29947 external_mini_cluster.cc:1428] Setting key 859b70416ecaac13b054d81ccfa19a61
I20260808 17:45:42.215546 2089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70da431546bb4303bc9314b2d827dda2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42881 } }
I20260808 17:45:42.215740 2089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:42.215806 2089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70da431546bb4303bc9314b2d827dda2, State: Initialized, Role: FOLLOWER
I20260808 17:45:42.215966 2089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70da431546bb4303bc9314b2d827dda2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42881 } }
I20260808 17:45:42.216065 2089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:42.216135 2089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:42.216220 2089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:42.216933 2089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70da431546bb4303bc9314b2d827dda2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42881 } }
I20260808 17:45:42.217068 2089 leader_election.cc:302] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70da431546bb4303bc9314b2d827dda2; no voters:
I20260808 17:45:42.217232 2089 leader_election.cc:288] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:42.217294 2094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:42.217442 2094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [term 1 LEADER]: Becoming Leader. State: Replica: 70da431546bb4303bc9314b2d827dda2, State: Running, Role: LEADER
I20260808 17:45:42.217548 2089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:42.217582 2094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70da431546bb4303bc9314b2d827dda2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42881 } }
I20260808 17:45:42.218387 2095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70da431546bb4303bc9314b2d827dda2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70da431546bb4303bc9314b2d827dda2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42881 } } }
I20260808 17:45:42.218425 2097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70da431546bb4303bc9314b2d827dda2. Latest consensus state: current_term: 1 leader_uuid: "70da431546bb4303bc9314b2d827dda2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70da431546bb4303bc9314b2d827dda2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42881 } } }
I20260808 17:45:42.218528 2097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:42.218709 2095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:42.218930 2108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:42.219440 2108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:42.220932 2108 catalog_manager.cc:1347] Generated new cluster ID: 2dcd4131613e4caeb995ccabf5a8c3dc
I20260808 17:45:42.220983 2108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:45:42 dist-test-slave-tk9z krb5kdc[1971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:42.228690 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:42.215985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43244 (local address 127.29.62.254:42881)
0808 17:45:42.216648 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:42.216656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:42.216683 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:45:42.216788 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:42.216794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:42.216836 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:42.216989 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:42.217843 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:42.219364 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:42.220452 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:42.221050 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:42.221173 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:42.224033 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:42.224038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:42.224038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:42.224051 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:42.226799 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:42.227214 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:42.227217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:42.227218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:42.227345 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:42.227566 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:42.227567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:42.227567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:42.227897 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:42.228073 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:42.228176 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 17:45:42.228445 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":566,"threads_started":1}
I20260808 17:45:42.229681 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:45:42.247651 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:42.243458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.29.62.254:42881)
0808 17:45:42.243511 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:42.243513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:42.243591 (+ 78us) server_negotiation.cc:407] Connection header received
0808 17:45:42.243644 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:42.243645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:42.243656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:42.243729 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:42.244156 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:42.244933 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:42.245934 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:42.246260 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:42.246345 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:42.247130 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:42.247131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:42.247132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:42.247379 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:42.247399 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:42.247406 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:42.247472 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:42.247539 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 17:45:42.247565 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:45:42.254355 2108 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:42.254891 2108 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:42.260293 2108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70da431546bb4303bc9314b2d827dda2: Generated new TSK 0
I20260808 17:45:42.260501 2108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:45:42.263999 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 1989
2026-08-08T17:45:42Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1558 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:42 dist-test-slave-tk9z krb5kdc[2130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:42 dist-test-slave-tk9z krb5kdc[2130](info): set up 2 sockets
Aug 08 17:45:42 dist-test-slave-tk9z krb5kdc[2130](info): commencing operation
krb5kdc: starting...
W20260808 17:45:43.497634 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:43 dist-test-slave-tk9z krb5kdc[2130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:43.623426 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:38117
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:38117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:43.707792 2149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:43.707994 2149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:43.708016 2149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:43.709162 2149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:43.709206 2149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:43.709220 2149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:43.709273 2149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:43.709307 2149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:43.710822 2149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:38117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:38117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:43.711099 2149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:43.711355 2149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:45:43.716708 2149 server_base.cc:1034] running on GCE node
W20260808 17:45:43.716722 2158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:43.716727 2154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:43.716882 2160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:43.717128 2149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:43.717387 2149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:43.718515 2149 hybrid_clock.cc:648] HybridClock initialized: now 1786211143718495 us; error 41 us; skew 500 ppm
Aug 08 17:45:43 dist-test-slave-tk9z krb5kdc[2130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211143, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:43.721026 2149 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:43.721413 2149 webserver.cc:460] Webserver started at http://127.29.62.254:43417/ using document root <none> and password file <none>
I20260808 17:45:43.721614 2149 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:43.721657 2149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:43.721746 2149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:43.722858 2149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "132dfccee7ff4a99a15a639240241ab0"
format_stamp: "Formatted at 2026-08-08 17:45:43 on dist-test-slave-tk9z"
server_key: "09648e280aaf1f298999757d5d3b2c37"
server_key_iv: "29da1ef0a99ddef7f66d6271baa01452"
server_key_version: "encryptionkey@0"
I20260808 17:45:43.723196 2149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "132dfccee7ff4a99a15a639240241ab0"
format_stamp: "Formatted at 2026-08-08 17:45:43 on dist-test-slave-tk9z"
server_key: "09648e280aaf1f298999757d5d3b2c37"
server_key_iv: "29da1ef0a99ddef7f66d6271baa01452"
server_key_version: "encryptionkey@0"
I20260808 17:45:43.724505 2149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:43.725206 2167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:43.725417 2149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:43.725528 2149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "132dfccee7ff4a99a15a639240241ab0"
format_stamp: "Formatted at 2026-08-08 17:45:43 on dist-test-slave-tk9z"
server_key: "09648e280aaf1f298999757d5d3b2c37"
server_key_iv: "29da1ef0a99ddef7f66d6271baa01452"
server_key_version: "encryptionkey@0"
I20260808 17:45:43.725615 2149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:43.735551 2149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:43.737362 2149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:43.737534 2149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:43.741446 2149 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:38117
I20260808 17:45:43.741498 2257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:38117 every 8 connection(s)
I20260808 17:45:43.741858 2149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:43.742946 2259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:43.745213 2259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0: Bootstrap starting.
I20260808 17:45:43.745803 2259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:43.746112 2259 log.cc:824] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:43.746876 2259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0: No bootstrap required, opened a new log
I20260808 17:45:43.748412 2259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132dfccee7ff4a99a15a639240241ab0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38117 } }
I20260808 17:45:43.748590 2259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:43.748636 2259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 132dfccee7ff4a99a15a639240241ab0, State: Initialized, Role: FOLLOWER
I20260808 17:45:43.748741 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 2149
I20260808 17:45:43.748848 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:43.748797 2259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132dfccee7ff4a99a15a639240241ab0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38117 } }
I20260808 17:45:43.748888 2259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:43.748942 2259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:43.748998 2259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:43.749059 29947 external_mini_cluster.cc:1428] Setting key 234ea40220853503a3b35f577711061d
I20260808 17:45:43.750068 2259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132dfccee7ff4a99a15a639240241ab0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38117 } }
I20260808 17:45:43.750195 2259 leader_election.cc:302] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 132dfccee7ff4a99a15a639240241ab0; no voters:
I20260808 17:45:43.750401 2259 leader_election.cc:288] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:43.750428 2265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:43.750604 2265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [term 1 LEADER]: Becoming Leader. State: Replica: 132dfccee7ff4a99a15a639240241ab0, State: Running, Role: LEADER
I20260808 17:45:43.750689 2259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:43.750726 2265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132dfccee7ff4a99a15a639240241ab0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38117 } }
I20260808 17:45:43.751092 2267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 132dfccee7ff4a99a15a639240241ab0. Latest consensus state: current_term: 1 leader_uuid: "132dfccee7ff4a99a15a639240241ab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132dfccee7ff4a99a15a639240241ab0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38117 } } }
I20260808 17:45:43.751104 2266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "132dfccee7ff4a99a15a639240241ab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "132dfccee7ff4a99a15a639240241ab0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38117 } } }
I20260808 17:45:43.751276 2267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:43.751283 2266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:43.751593 2275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:43.752233 2275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:43.753717 2275 catalog_manager.cc:1347] Generated new cluster ID: 11a08c1f0e024061965ebb67f408bb40
I20260808 17:45:43.753773 2275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:45:43 dist-test-slave-tk9z krb5kdc[2130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:43.760856 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:43.749583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.29.62.254:38117)
0808 17:45:43.749773 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:43.749782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:43.749816 (+ 34us) server_negotiation.cc:407] Connection header received
0808 17:45:43.749910 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:43.749915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:43.749955 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:43.750066 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:43.751027 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:43.752517 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:43.753453 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:43.754005 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:43.754130 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:43.756659 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:43.756664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:43.756664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:43.756676 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:43.758591 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:43.759213 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:43.759217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:43.759217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:43.759369 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:43.759569 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:43.759569 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:43.759570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:43.759850 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:43.759998 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:43.760512 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 17:45:43.760658 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260808 17:45:43.770448 2275 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:43.771129 2275 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:43.774996 2275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 132dfccee7ff4a99a15a639240241ab0: Generated new TSK 0
I20260808 17:45:43.775158 2275 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:43.807296 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38117
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:43.889601 2294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:43.889804 2294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:43.889829 2294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:43.891088 2294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:43.891134 2294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:43.891188 2294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:43.892737 2294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:43.892990 2294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:43.893234 2294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:43.898340 2305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:43.898370 2301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:43.898340 2303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:43.898808 2294 server_base.cc:1034] running on GCE node
I20260808 17:45:43.898980 2294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:43.899222 2294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:43.900379 2294 hybrid_clock.cc:648] HybridClock initialized: now 1786211143900360 us; error 29 us; skew 500 ppm
Aug 08 17:45:43 dist-test-slave-tk9z krb5kdc[2130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211143, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:43.903028 2294 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:43.903505 2294 webserver.cc:460] Webserver started at http://127.29.62.193:39493/ using document root <none> and password file <none>
I20260808 17:45:43.903750 2294 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:43.903821 2294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:43.903985 2294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:43.905042 2294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "f98af3cb034741878bbbf8f5dac28ea3"
format_stamp: "Formatted at 2026-08-08 17:45:43 on dist-test-slave-tk9z"
server_key: "041d61c98cbd9c5835dad3fc0b109e63"
server_key_iv: "4c5ed1ebb859963db31d33ea68afc9eb"
server_key_version: "encryptionkey@0"
I20260808 17:45:43.905434 2294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "f98af3cb034741878bbbf8f5dac28ea3"
format_stamp: "Formatted at 2026-08-08 17:45:43 on dist-test-slave-tk9z"
server_key: "041d61c98cbd9c5835dad3fc0b109e63"
server_key_iv: "4c5ed1ebb859963db31d33ea68afc9eb"
server_key_version: "encryptionkey@0"
I20260808 17:45:43.906919 2294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:43.907671 2319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:43.907845 2294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:45:43.907953 2294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "f98af3cb034741878bbbf8f5dac28ea3"
format_stamp: "Formatted at 2026-08-08 17:45:43 on dist-test-slave-tk9z"
server_key: "041d61c98cbd9c5835dad3fc0b109e63"
server_key_iv: "4c5ed1ebb859963db31d33ea68afc9eb"
server_key_version: "encryptionkey@0"
I20260808 17:45:43.908036 2294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:43.939193 2294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:43.941222 2294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:43.941409 2294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:43.941679 2294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:43.942026 2294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:43.942090 2294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:43.942131 2294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:43.942160 2294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:43.948329 2294 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:38841
I20260808 17:45:43.948381 2493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:38841 every 8 connection(s)
I20260808 17:45:43.948710 2294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:43.952955 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 2294
I20260808 17:45:43.953085 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:43.953364 29947 external_mini_cluster.cc:1428] Setting key 2e374be3a697b6721ff0f9d6213ab449
Aug 08 17:45:43 dist-test-slave-tk9z krb5kdc[2130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211143, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:43.959301 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:43.949470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:47163 (local address 127.29.62.254:38117)
0808 17:45:43.949551 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:43.949553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:43.949787 (+ 234us) server_negotiation.cc:407] Connection header received
0808 17:45:43.950295 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:43.950296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:43.950311 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:43.950367 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:43.951491 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:43.952188 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:43.953271 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:43.953576 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:43.953679 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:43.956111 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:43.956113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:43.956113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:43.956122 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:43.957793 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:43.958200 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:43.958202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:43.958202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:43.958272 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:43.958587 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:43.958588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:43.958589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:43.958754 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:43.958835 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:43.959110 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 17:45:43.959211 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 17:45:43.959470 2496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:43.949467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38117 (local address 127.29.62.193:47163)
0808 17:45:43.949649 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:43.949688 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:43.950190 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:43.950519 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:43.950523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:43.951203 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:43.951400 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:43.951405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:43.952270 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:43.952271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:43.953155 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:43.953177 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:43.953687 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:43.953688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:43.953829 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:43.954357 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:43.954411 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:43.956030 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:43.957882 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:43.957888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:43.957890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:43.958117 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:43.958342 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:43.958342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:43.958343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:43.958507 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:43.958830 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:43.958835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:43.958950 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 17:45:43.959068 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 17:45:43.959231 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260808 17:45:43.959950 2494 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38117
I20260808 17:45:43.960071 2494 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:43.960287 2494 heartbeater.cc:511] Master 127.29.62.254:38117 requested a full tablet report, sending...
I20260808 17:45:43.960846 2198 ts_manager.cc:194] Registered new tserver with Master: f98af3cb034741878bbbf8f5dac28ea3 (127.29.62.193:38841)
I20260808 17:45:43.961463 2198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:47163
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:43.981487 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38117
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:44.065955 2500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:44.066149 2500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:44.066174 2500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:44.067335 2500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:44.067376 2500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:44.067407 2500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:44.068776 2500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:44.068948 2500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:44.069136 2500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:44.074175 2505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:44.074201 2508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:44.074229 2506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:44.074465 2500 server_base.cc:1034] running on GCE node
I20260808 17:45:44.074689 2500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:44.074931 2500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:44.076114 2500 hybrid_clock.cc:648] HybridClock initialized: now 1786211144076072 us; error 31 us; skew 500 ppm
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:44.078660 2500 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:44.079123 2500 webserver.cc:460] Webserver started at http://127.29.62.194:43819/ using document root <none> and password file <none>
I20260808 17:45:44.079384 2500 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:44.079461 2500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:44.079630 2500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:44.080700 2500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "119c91fcde4b43cbb43b795c0d02dfc0"
format_stamp: "Formatted at 2026-08-08 17:45:44 on dist-test-slave-tk9z"
server_key: "70d5ccf51732b1681e30e93c24b2a910"
server_key_iv: "c9235dd42d0009ac248c53b106ee8b0b"
server_key_version: "encryptionkey@0"
I20260808 17:45:44.081090 2500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "119c91fcde4b43cbb43b795c0d02dfc0"
format_stamp: "Formatted at 2026-08-08 17:45:44 on dist-test-slave-tk9z"
server_key: "70d5ccf51732b1681e30e93c24b2a910"
server_key_iv: "c9235dd42d0009ac248c53b106ee8b0b"
server_key_version: "encryptionkey@0"
I20260808 17:45:44.082520 2500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:45:44.083300 2517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:44.083503 2500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:44.083622 2500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "119c91fcde4b43cbb43b795c0d02dfc0"
format_stamp: "Formatted at 2026-08-08 17:45:44 on dist-test-slave-tk9z"
server_key: "70d5ccf51732b1681e30e93c24b2a910"
server_key_iv: "c9235dd42d0009ac248c53b106ee8b0b"
server_key_version: "encryptionkey@0"
I20260808 17:45:44.083707 2500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:44.101040 2500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:44.103158 2500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:44.103339 2500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:44.103592 2500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:44.103945 2500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:44.104007 2500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:44.104051 2500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:44.104081 2500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:44.110307 2500 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:45023
I20260808 17:45:44.110438 2690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:45023 every 8 connection(s)
I20260808 17:45:44.110903 2500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:44.117105 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 2500
I20260808 17:45:44.117249 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:44.117479 29947 external_mini_cluster.cc:1428] Setting key 5affe6df3d189b42341ac3160e98833a
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:44.121537 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:44.111812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:43185 (local address 127.29.62.254:38117)
0808 17:45:44.111866 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:44.111868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:44.112136 (+ 268us) server_negotiation.cc:407] Connection header received
0808 17:45:44.112623 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:44.112624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:44.112635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:44.112706 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:44.113654 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.114203 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.115213 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.115479 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.115542 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:44.118121 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:44.118122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:44.118123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:44.118155 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:44.120026 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:44.120518 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:44.120520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:44.120520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:44.120584 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:44.120857 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:44.120858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:44.120859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:44.120980 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:44.121065 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:44.121352 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 17:45:44.121451 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:45:44.121721 2694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:44.111773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38117 (local address 127.29.62.194:43185)
0808 17:45:44.112024 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:44.112055 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:44.112535 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:44.112762 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:44.112765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:44.113355 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:44.113553 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:44.113558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.114318 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.114318 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:44.115132 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:44.115134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.115568 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.115568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:44.115668 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:44.116202 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:44.116210 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:44.118027 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:44.120125 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:44.120131 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:44.120133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:44.120413 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:44.120658 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:44.120659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:44.120659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:44.120779 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:44.121057 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:44.121064 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:44.121206 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 17:45:44.121316 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 17:45:44.121470 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":90,"threads_started":1}
I20260808 17:45:44.122298 2691 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38117
I20260808 17:45:44.122448 2691 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:44.122653 2691 heartbeater.cc:511] Master 127.29.62.254:38117 requested a full tablet report, sending...
I20260808 17:45:44.123164 2198 ts_manager.cc:194] Registered new tserver with Master: 119c91fcde4b43cbb43b795c0d02dfc0 (127.29.62.194:45023)
I20260808 17:45:44.123657 2198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:43185
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:44.145484 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38117
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:44.227769 2701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:44.228066 2701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:44.228112 2701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:44.229405 2701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:44.229480 2701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:44.229529 2701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:44.231114 2701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:44.231382 2701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:44.231629 2701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:44.236840 2707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:44.236846 2709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:44.237017 2701 server_base.cc:1034] running on GCE node
W20260808 17:45:44.237196 2711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:44.237414 2701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:44.237694 2701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:44.238867 2701 hybrid_clock.cc:648] HybridClock initialized: now 1786211144238852 us; error 45 us; skew 500 ppm
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:44.241422 2701 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:44.241847 2701 webserver.cc:460] Webserver started at http://127.29.62.195:41399/ using document root <none> and password file <none>
I20260808 17:45:44.242101 2701 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:44.242177 2701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:44.242333 2701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:44.243441 2701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "aa347632c6af46b3ae7ffc61a8265e79"
format_stamp: "Formatted at 2026-08-08 17:45:44 on dist-test-slave-tk9z"
server_key: "f6a452b400f78882022f21ff64b2073c"
server_key_iv: "6314154be979e13b0b991765d2bfd936"
server_key_version: "encryptionkey@0"
I20260808 17:45:44.243827 2701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "aa347632c6af46b3ae7ffc61a8265e79"
format_stamp: "Formatted at 2026-08-08 17:45:44 on dist-test-slave-tk9z"
server_key: "f6a452b400f78882022f21ff64b2073c"
server_key_iv: "6314154be979e13b0b991765d2bfd936"
server_key_version: "encryptionkey@0"
I20260808 17:45:44.245180 2701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:44.245939 2724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:44.246119 2701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:44.246223 2701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "aa347632c6af46b3ae7ffc61a8265e79"
format_stamp: "Formatted at 2026-08-08 17:45:44 on dist-test-slave-tk9z"
server_key: "f6a452b400f78882022f21ff64b2073c"
server_key_iv: "6314154be979e13b0b991765d2bfd936"
server_key_version: "encryptionkey@0"
I20260808 17:45:44.246306 2701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:44.272192 2701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:44.274217 2701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:44.274463 2701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:44.274753 2701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:44.275126 2701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:44.275188 2701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:44.275233 2701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:44.275254 2701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:44.281418 2701 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:45903
I20260808 17:45:44.281462 2907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:45903 every 8 connection(s)
I20260808 17:45:44.281800 2701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:44.291213 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 2701
I20260808 17:45:44.291319 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:44.291527 29947 external_mini_cluster.cc:1428] Setting key dc8e789e2adda2a828050bd54e982d16
I20260808 17:45:44.291535 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:44.282646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:47763 (local address 127.29.62.254:38117)
0808 17:45:44.282721 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:44.282723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:44.282929 (+ 206us) server_negotiation.cc:407] Connection header received
0808 17:45:44.283433 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:44.283434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:44.283444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:44.283517 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:44.284431 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.284927 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.285895 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.286202 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.286257 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:44.288539 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:44.288540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:44.288541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:44.288548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:44.290060 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:44.290538 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:44.290539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:44.290540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:44.290598 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:44.290825 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:44.290826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:44.290826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:44.290936 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:44.290987 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:44.291367 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 17:45:44.291455 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:45:44.291786 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:44.282627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38117 (local address 127.29.62.195:47763)
0808 17:45:44.282799 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:44.282843 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:44.283344 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:44.283571 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:44.283574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:44.284144 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:44.284343 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:44.284348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.285013 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.285013 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:44.285808 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:44.285810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.286267 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.286268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:44.286433 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:44.286947 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:44.286956 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:44.288434 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:44.290139 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:44.290146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:44.290148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:44.290465 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:44.290662 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:44.290662 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:44.290663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:44.290767 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:44.290995 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:44.291000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:44.291197 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 17:45:44.291326 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 17:45:44.291530 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260808 17:45:44.292259 2911 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38117
I20260808 17:45:44.292378 2911 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:44.292600 2911 heartbeater.cc:511] Master 127.29.62.254:38117 requested a full tablet report, sending...
I20260808 17:45:44.293035 2198 ts_manager.cc:194] Registered new tserver with Master: aa347632c6af46b3ae7ffc61a8265e79 (127.29.62.195:45903)
I20260808 17:45:44.293419 2198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:47763
I20260808 17:45:44.303603 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211144, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211144, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:44.327719 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:44.320569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.29.62.254:38117)
0808 17:45:44.320654 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:44.320656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:44.320754 (+ 98us) server_negotiation.cc:407] Connection header received
0808 17:45:44.320810 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:44.320811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:44.320821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:44.320865 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:44.321404 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.321896 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.322725 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:44.322906 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:44.322960 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:44.325159 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:44.325160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:44.325161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:44.325167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:44.326689 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:44.326978 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:44.326978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:44.326979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:44.327036 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:44.327183 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:44.327184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:44.327184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:44.327319 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:44.327390 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:44.327553 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 17:45:44.327614 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 17:45:44.327970 2198 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38620
I20260808 17:45:44.328722 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 2294
I20260808 17:45:44.344586 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 2500
I20260808 17:45:44.362516 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 2701
I20260808 17:45:44.369427 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 2149
2026-08-08T17:45:44Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2099 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2927](info): set up 2 sockets
Aug 08 17:45:44 dist-test-slave-tk9z krb5kdc[2927](info): commencing operation
krb5kdc: starting...
W20260808 17:45:45.638648 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:45 dist-test-slave-tk9z krb5kdc[2927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:45.762831 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:44425
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:44425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:45.844619 2950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:45.844862 2950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:45.844899 2950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:45.846032 2950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:45.846104 2950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:45.846129 2950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:45.846163 2950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:45.846189 2950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:45.847767 2950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:44425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:44425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:45.848049 2950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:45.848279 2950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:45.853358 2961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:45.853363 2962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:45.853593 2950 server_base.cc:1034] running on GCE node
W20260808 17:45:45.853391 2965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:45.853914 2950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:45.854190 2950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:45.855363 2950 hybrid_clock.cc:648] HybridClock initialized: now 1786211145855333 us; error 48 us; skew 500 ppm
Aug 08 17:45:45 dist-test-slave-tk9z krb5kdc[2927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211145, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:45.857906 2950 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:45.858438 2950 webserver.cc:460] Webserver started at http://127.29.62.254:32879/ using document root <none> and password file <none>
I20260808 17:45:45.858671 2950 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:45.858739 2950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:45.858896 2950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:45.859985 2950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "b86bf370a23c46e4b4d61be6e7fb8e45"
format_stamp: "Formatted at 2026-08-08 17:45:45 on dist-test-slave-tk9z"
server_key: "d1b52e182f6c9a98c0ea03c0b4168feb"
server_key_iv: "4e78282df13ad68966e83abf57fc9426"
server_key_version: "encryptionkey@0"
I20260808 17:45:45.860376 2950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "b86bf370a23c46e4b4d61be6e7fb8e45"
format_stamp: "Formatted at 2026-08-08 17:45:45 on dist-test-slave-tk9z"
server_key: "d1b52e182f6c9a98c0ea03c0b4168feb"
server_key_iv: "4e78282df13ad68966e83abf57fc9426"
server_key_version: "encryptionkey@0"
I20260808 17:45:45.861754 2950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:45.862587 2975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:45.862768 2950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:45.862881 2950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "b86bf370a23c46e4b4d61be6e7fb8e45"
format_stamp: "Formatted at 2026-08-08 17:45:45 on dist-test-slave-tk9z"
server_key: "d1b52e182f6c9a98c0ea03c0b4168feb"
server_key_iv: "4e78282df13ad68966e83abf57fc9426"
server_key_version: "encryptionkey@0"
I20260808 17:45:45.862959 2950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:45.872442 2950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:45.874825 2950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:45.875003 2950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:45.879329 2950 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:44425
I20260808 17:45:45.879583 3060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:44425 every 8 connection(s)
I20260808 17:45:45.879765 2950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:45.880481 3065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:45.882884 3065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45: Bootstrap starting.
I20260808 17:45:45.883502 3065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:45.883817 3065 log.cc:824] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:45.884541 3065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45: No bootstrap required, opened a new log
I20260808 17:45:45.886097 3065 raft_consensus.cc:348] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86bf370a23c46e4b4d61be6e7fb8e45" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44425 } }
I20260808 17:45:45.886237 3065 raft_consensus.cc:374] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:45.886291 3065 raft_consensus.cc:729] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86bf370a23c46e4b4d61be6e7fb8e45, State: Initialized, Role: FOLLOWER
I20260808 17:45:45.886480 3065 consensus_queue.cc:260] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86bf370a23c46e4b4d61be6e7fb8e45" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44425 } }
I20260808 17:45:45.886572 3065 raft_consensus.cc:388] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:45.886623 3065 raft_consensus.cc:482] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:45.886672 3065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:45.887498 3065 raft_consensus.cc:504] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86bf370a23c46e4b4d61be6e7fb8e45" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44425 } }
I20260808 17:45:45.887620 3065 leader_election.cc:302] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b86bf370a23c46e4b4d61be6e7fb8e45; no voters:
I20260808 17:45:45.887809 3065 leader_election.cc:288] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:45.887858 3071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:45.888042 3071 raft_consensus.cc:686] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [term 1 LEADER]: Becoming Leader. State: Replica: b86bf370a23c46e4b4d61be6e7fb8e45, State: Running, Role: LEADER
I20260808 17:45:45.888113 3065 sys_catalog.cc:564] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:45.888098 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 2950
I20260808 17:45:45.888185 3071 consensus_queue.cc:237] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86bf370a23c46e4b4d61be6e7fb8e45" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44425 } }
I20260808 17:45:45.888230 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:45.888435 29947 external_mini_cluster.cc:1428] Setting key fb9f04320546b0b2eac029ea9e3ca5c1
I20260808 17:45:45.888624 3073 sys_catalog.cc:455] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b86bf370a23c46e4b4d61be6e7fb8e45. Latest consensus state: current_term: 1 leader_uuid: "b86bf370a23c46e4b4d61be6e7fb8e45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86bf370a23c46e4b4d61be6e7fb8e45" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44425 } } }
I20260808 17:45:45.888712 3073 sys_catalog.cc:458] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:45.889164 3072 sys_catalog.cc:455] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b86bf370a23c46e4b4d61be6e7fb8e45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86bf370a23c46e4b4d61be6e7fb8e45" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44425 } } }
I20260808 17:45:45.889246 3072 sys_catalog.cc:458] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45 [sys.catalog]: This master's current role is: LEADER
W20260808 17:45:45.890131 3099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:45:45.890216 3099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:45:45.890295 3079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:45.890494 3079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:45.892128 3079 catalog_manager.cc:1347] Generated new cluster ID: 9e12a5f654e549ee8bfe3039dee377b9
I20260808 17:45:45.892174 3079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:45:45 dist-test-slave-tk9z krb5kdc[2927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:45.899605 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:45.888919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.29.62.254:44425)
0808 17:45:45.889082 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:45.889090 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:45.889524 (+ 434us) server_negotiation.cc:407] Connection header received
0808 17:45:45.889766 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:45.889774 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:45.889833 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:45.890003 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:45.890921 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:45.892130 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:45.892932 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:45.893430 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:45.893496 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:45.895970 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:45.895974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:45.895975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:45.895988 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:45.898008 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:45.898424 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:45.898427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:45.898428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:45.898525 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:45.898753 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:45.898754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:45.898754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:45.898987 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:45.899170 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:45.899253 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 17:45:45.899394 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260808 17:45:45.920611 3079 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:45.921144 3079 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:45.927518 3079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b86bf370a23c46e4b4d61be6e7fb8e45: Generated new TSK 0
I20260808 17:45:45.927711 3079 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:45.964871 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:44425
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:46.048687 3110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:46.048914 3110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:46.048956 3110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:46.050282 3110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:46.050355 3110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:46.050414 3110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:46.051931 3110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:44425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:46.052211 3110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:46.052440 3110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:46.057550 3120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:46.057598 3110 server_base.cc:1034] running on GCE node
W20260808 17:45:46.057585 3119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:46.057569 3122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:46.057967 3110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:46.058238 3110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:46.059408 3110 hybrid_clock.cc:648] HybridClock initialized: now 1786211146059405 us; error 43 us; skew 500 ppm
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:46.061980 3110 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:46.062492 3110 webserver.cc:460] Webserver started at http://127.29.62.193:38797/ using document root <none> and password file <none>
I20260808 17:45:46.062698 3110 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:46.062740 3110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:46.062831 3110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:46.063937 3110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "8a81f08a739b4c689e4ad390163bea10"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "f4491ac8269151e91f56e65c55807102"
server_key_iv: "8049facfc6b0b68a0531141298712135"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.064285 3110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "8a81f08a739b4c689e4ad390163bea10"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "f4491ac8269151e91f56e65c55807102"
server_key_iv: "8049facfc6b0b68a0531141298712135"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.065610 3110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:46.066417 3138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.066639 3110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:46.066766 3110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "8a81f08a739b4c689e4ad390163bea10"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "f4491ac8269151e91f56e65c55807102"
server_key_iv: "8049facfc6b0b68a0531141298712135"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.066847 3110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:46.080297 3110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:46.082885 3110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:46.083010 3110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:46.083194 3110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:46.083630 3110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:46.083673 3110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.083732 3110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:46.083760 3110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.090549 3110 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:38559
I20260808 17:45:46.090598 3314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:38559 every 8 connection(s)
I20260808 17:45:46.090929 3110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:46.100378 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:46.091670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:38393 (local address 127.29.62.254:44425)
0808 17:45:46.091722 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:46.091724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:46.091917 (+ 193us) server_negotiation.cc:407] Connection header received
0808 17:45:46.092851 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:46.092852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:46.092866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:46.092926 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:46.093494 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.094000 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.095252 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.095476 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.095537 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:46.097568 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:46.097570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:46.097570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:46.097577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:46.099129 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.099524 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.099525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.099525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.099582 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.099813 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.099814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.099814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.099920 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:46.099972 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:46.100203 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 17:45:46.100311 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:45:46.100472 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:46.091606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:44425 (local address 127.29.62.193:38393)
0808 17:45:46.091781 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:46.091824 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:46.092749 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:46.092980 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:46.092984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:46.093223 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:46.093416 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:46.093420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.094103 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.094104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:46.095161 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:46.095164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.095561 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.095561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:46.095661 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:46.096117 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:46.096125 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:46.097475 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:46.099212 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:46.099218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:46.099220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:46.099443 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:46.099651 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:46.099652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:46.099652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:46.099748 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:46.099980 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:46.099984 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:46.100089 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 17:45:46.100167 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:45:46.100278 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260808 17:45:46.100677 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 3110
I20260808 17:45:46.100778 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:46.100961 29947 external_mini_cluster.cc:1428] Setting key de6330e20cbb7bc3357ccc767faa5b28
I20260808 17:45:46.101168 3315 heartbeater.cc:347] Connected to a master server at 127.29.62.254:44425
I20260808 17:45:46.101279 3315 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:46.101483 3315 heartbeater.cc:511] Master 127.29.62.254:44425 requested a full tablet report, sending...
I20260808 17:45:46.102084 3003 ts_manager.cc:194] Registered new tserver with Master: 8a81f08a739b4c689e4ad390163bea10 (127.29.62.193:38559)
I20260808 17:45:46.102999 3003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:38393
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:46.129179 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:44425
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:46.213004 3322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:46.213261 3322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:46.213301 3322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:46.214522 3322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:46.214599 3322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:46.214644 3322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:46.216176 3322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:44425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:46.216468 3322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:46.216706 3322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:46.221585 3332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:46.221585 3329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:46.221617 3328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:46.221957 3322 server_base.cc:1034] running on GCE node
I20260808 17:45:46.222213 3322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:46.222546 3322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:46.223724 3322 hybrid_clock.cc:648] HybridClock initialized: now 1786211146223715 us; error 37 us; skew 500 ppm
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:46.226627 3322 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:46.227144 3322 webserver.cc:460] Webserver started at http://127.29.62.194:41617/ using document root <none> and password file <none>
I20260808 17:45:46.227384 3322 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:46.227465 3322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:46.227618 3322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:46.228912 3322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "59aa0fa360c440ba80b04345f48fe3ec"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "205120782dbf400d71696a1adbc41ae7"
server_key_iv: "c62fda63f84dbc2b7dead25216c2b8ca"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.229334 3322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "59aa0fa360c440ba80b04345f48fe3ec"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "205120782dbf400d71696a1adbc41ae7"
server_key_iv: "c62fda63f84dbc2b7dead25216c2b8ca"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.230863 3322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 17:45:46.231726 3341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.231928 3322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:46.232039 3322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "59aa0fa360c440ba80b04345f48fe3ec"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "205120782dbf400d71696a1adbc41ae7"
server_key_iv: "c62fda63f84dbc2b7dead25216c2b8ca"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.232116 3322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:46.242604 3322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:46.244890 3322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:46.245059 3322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:46.245290 3322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:46.245616 3322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:46.245672 3322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.245708 3322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:46.245736 3322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.252489 3322 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:33813
I20260808 17:45:46.252533 3515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:33813 every 8 connection(s)
I20260808 17:45:46.252853 3322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:46.254055 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 3322
I20260808 17:45:46.254168 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:46.254356 29947 external_mini_cluster.cc:1428] Setting key 0a7b0a5207956a275b434030f1ee30cd
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
I20260808 17:45:46.263149 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:46.253551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:54039 (local address 127.29.62.254:44425)
0808 17:45:46.253611 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:46.253612 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:46.253824 (+ 212us) server_negotiation.cc:407] Connection header received
0808 17:45:46.255220 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:46.255221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:46.255231 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:46.255280 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:46.256098 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.256579 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.257705 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.257923 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.257998 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:46.260288 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:46.260289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:46.260290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:46.260297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:46.261695 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.262103 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.262105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.262105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.262170 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.262459 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.262460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.262461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.262585 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:46.262663 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:46.262962 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 17:45:46.263061 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:45:46.263303 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:46.253508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:44425 (local address 127.29.62.194:54039)
0808 17:45:46.253663 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:46.253704 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:46.255105 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:46.255358 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:46.255362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:46.255729 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:46.256012 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:46.256018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.256673 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.256674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:46.257600 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:46.257603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.258005 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.258006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:46.258148 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:46.258712 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:46.258721 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:46.260180 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:46.261789 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:46.261797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:46.261799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:46.262021 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:46.262238 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:46.262239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:46.262240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:46.262363 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:46.262655 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:46.262662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:46.262823 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 17:45:46.262924 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 17:45:46.263077 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260808 17:45:46.263785 3516 heartbeater.cc:347] Connected to a master server at 127.29.62.254:44425
I20260808 17:45:46.263896 3516 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:46.264115 3516 heartbeater.cc:511] Master 127.29.62.254:44425 requested a full tablet report, sending...
I20260808 17:45:46.264528 3003 ts_manager.cc:194] Registered new tserver with Master: 59aa0fa360c440ba80b04345f48fe3ec (127.29.62.194:33813)
I20260808 17:45:46.264863 3003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:54039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:46.280881 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:44425
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:46.370039 3525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:46.370338 3525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:46.370407 3525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:46.371685 3525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:46.371752 3525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:46.371801 3525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:46.373454 3525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:44425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:46.373729 3525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:46.373975 3525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:46.379421 3531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:46.379421 3535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:46.379451 3533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:46.379802 3525 server_base.cc:1034] running on GCE node
I20260808 17:45:46.380026 3525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:46.380236 3525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:46.381393 3525 hybrid_clock.cc:648] HybridClock initialized: now 1786211146381386 us; error 30 us; skew 500 ppm
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:46.384315 3525 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:46.384837 3525 webserver.cc:460] Webserver started at http://127.29.62.195:39655/ using document root <none> and password file <none>
I20260808 17:45:46.385041 3525 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:46.385085 3525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:46.385186 3525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:46.386297 3525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "f187cb99a3a942ef8aee62e65e25daf1"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "8b49987faafa7495a344af3d3b0ab5c4"
server_key_iv: "1b4fcae70bc8a26f3cb6fd35148570fc"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.386796 3525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "f187cb99a3a942ef8aee62e65e25daf1"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "8b49987faafa7495a344af3d3b0ab5c4"
server_key_iv: "1b4fcae70bc8a26f3cb6fd35148570fc"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.388406 3525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:46.389371 3544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.389575 3525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:46.389663 3525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "f187cb99a3a942ef8aee62e65e25daf1"
format_stamp: "Formatted at 2026-08-08 17:45:46 on dist-test-slave-tk9z"
server_key: "8b49987faafa7495a344af3d3b0ab5c4"
server_key_iv: "1b4fcae70bc8a26f3cb6fd35148570fc"
server_key_version: "encryptionkey@0"
I20260808 17:45:46.389722 3525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:46.403402 3525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:46.405750 3525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:46.405879 3525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:46.406080 3525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:46.406508 3525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:46.406550 3525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.406574 3525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:46.406618 3525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:46.412604 3525 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:32961
I20260808 17:45:46.412662 3727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:32961 every 8 connection(s)
I20260808 17:45:46.412997 3525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:45:46.416373 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 3525
I20260808 17:45:46.416488 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:46.416667 29947 external_mini_cluster.cc:1428] Setting key a163b25580d05ebf896e851711209fee
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:46.422768 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:46.413679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:36479 (local address 127.29.62.254:44425)
0808 17:45:46.413769 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:46.413771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:46.413955 (+ 184us) server_negotiation.cc:407] Connection header received
0808 17:45:46.415002 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:46.415003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:46.415013 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:46.415070 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:46.415689 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.416208 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.417366 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.417568 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.417656 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:46.419842 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:46.419843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:46.419843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:46.419852 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:46.421193 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.421667 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.421668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.421669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.421736 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.422013 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.422015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.422015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.422152 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:46.422251 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:46.422589 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 17:45:46.422671 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 17:45:46.422971 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:46.413645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:44425 (local address 127.29.62.195:36479)
0808 17:45:46.413828 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:46.413863 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:46.414902 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:46.415135 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:46.415139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:46.415424 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:46.415616 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:46.415620 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.416323 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.416324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:46.417259 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:46.417262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.417667 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.417668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:46.417805 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:46.418257 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:46.418268 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:46.419738 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:46.421319 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:46.421327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:46.421329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:46.421568 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:46.421827 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:46.421828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:46.421829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:46.421937 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:46.422244 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:46.422251 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:46.422439 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 17:45:46.422563 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 17:45:46.422723 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260808 17:45:46.423441 3728 heartbeater.cc:347] Connected to a master server at 127.29.62.254:44425
I20260808 17:45:46.423547 3728 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:46.423732 3728 heartbeater.cc:511] Master 127.29.62.254:44425 requested a full tablet report, sending...
I20260808 17:45:46.424101 3003 ts_manager.cc:194] Registered new tserver with Master: f187cb99a3a942ef8aee62e65e25daf1 (127.29.62.195:32961)
I20260808 17:45:46.424441 3003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:36479
I20260808 17:45:46.428328 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:46.452860 3738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:46.444957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.29.62.193:38559)
0808 17:45:46.445182 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:46.445187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:46.445198 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:45:46.445227 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:46.445229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:46.445244 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:46.445313 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:46.445887 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.446682 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.447366 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.447686 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.447739 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:46.449953 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:46.449966 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:46.449966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:46.449977 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:46.451623 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.452006 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.452010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.452010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.452074 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.452279 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.452280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.452280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.452430 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:46.452593 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:46.452647 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 17:45:46.452727 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":128,"threads_started":1}
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[2927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:46.460626 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:46.454261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52678 (local address 127.29.62.254:44425)
0808 17:45:46.454332 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:46.454334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:46.454343 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:46.454497 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:46.454498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:46.454508 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:46.454555 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:46.454973 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.455456 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.456077 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:46.456253 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:46.456302 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:46.458424 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:46.458426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:46.458426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:46.458434 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:46.459651 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.459959 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.459960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.459960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.460014 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:46.460185 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:46.460185 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:46.460185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:46.460326 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:46.460402 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:46.460500 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 17:45:46.460563 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 17:45:46.460906 3003 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52678
I20260808 17:45:46.461278 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 3110
I20260808 17:45:46.480278 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 3322
I20260808 17:45:46.487254 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 3525
I20260808 17:45:46.507413 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 2950
2026-08-08T17:45:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2137 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[3744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[3744](info): set up 2 sockets
Aug 08 17:45:46 dist-test-slave-tk9z krb5kdc[3744](info): commencing operation
krb5kdc: starting...
W20260808 17:45:47.741355 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:47 dist-test-slave-tk9z krb5kdc[3744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:47.864993 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40933
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:47.948498 3764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:47.948751 3764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:47.948792 3764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:47.949965 3764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:47.950042 3764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:47.950067 3764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:47.950089 3764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:47.950120 3764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:47.951709 3764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:47.952003 3764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:47.952248 3764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:47.957132 3770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:47.957134 3773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:47.957156 3769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:47.957388 3764 server_base.cc:1034] running on GCE node
I20260808 17:45:47.957626 3764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:47.957865 3764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:47.959071 3764 hybrid_clock.cc:648] HybridClock initialized: now 1786211147959058 us; error 30 us; skew 500 ppm
Aug 08 17:45:47 dist-test-slave-tk9z krb5kdc[3744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211147, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:47.966329 3764 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:47.966780 3764 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:47.966861 3764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:47.967025 3764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:47.968164 3764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "9ef44163d1d44cb8ae277f96f7b492ee"
format_stamp: "Formatted at 2026-08-08 17:45:47 on dist-test-slave-tk9z"
server_key: "89e3ba416ed626e23ad95ee9efa834a7"
server_key_iv: "b586aa1c3bba2a0b6e973d9ff3a4eb2e"
server_key_version: "encryptionkey@0"
I20260808 17:45:47.968562 3764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "9ef44163d1d44cb8ae277f96f7b492ee"
format_stamp: "Formatted at 2026-08-08 17:45:47 on dist-test-slave-tk9z"
server_key: "89e3ba416ed626e23ad95ee9efa834a7"
server_key_iv: "b586aa1c3bba2a0b6e973d9ff3a4eb2e"
server_key_version: "encryptionkey@0"
I20260808 17:45:47.969950 3764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:47.970826 3782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:47.971029 3764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:45:47.971145 3764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "9ef44163d1d44cb8ae277f96f7b492ee"
format_stamp: "Formatted at 2026-08-08 17:45:47 on dist-test-slave-tk9z"
server_key: "89e3ba416ed626e23ad95ee9efa834a7"
server_key_iv: "b586aa1c3bba2a0b6e973d9ff3a4eb2e"
server_key_version: "encryptionkey@0"
I20260808 17:45:47.971235 3764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:47.987593 3764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:47.989528 3764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:47.989710 3764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:47.993960 3764 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40933
I20260808 17:45:47.993992 3857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40933 every 8 connection(s)
I20260808 17:45:47.994305 3764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:47.995477 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:47.997792 3859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee: Bootstrap starting.
I20260808 17:45:47.998345 3859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:47.998685 3859 log.cc:824] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:47.999404 3859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee: No bootstrap required, opened a new log
I20260808 17:45:48.001066 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 3764
I20260808 17:45:48.001185 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:48.001030 3859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ef44163d1d44cb8ae277f96f7b492ee" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40933 } }
I20260808 17:45:48.001219 3859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:48.001255 3859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef44163d1d44cb8ae277f96f7b492ee, State: Initialized, Role: FOLLOWER
I20260808 17:45:48.001380 29947 external_mini_cluster.cc:1428] Setting key a3c9906b44fc0cc810f374c3c5821e8d
I20260808 17:45:48.001410 3859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ef44163d1d44cb8ae277f96f7b492ee" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40933 } }
I20260808 17:45:48.001498 3859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:48.001538 3859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:48.001593 3859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:48.002339 3859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ef44163d1d44cb8ae277f96f7b492ee" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40933 } }
I20260808 17:45:48.002497 3859 leader_election.cc:302] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ef44163d1d44cb8ae277f96f7b492ee; no voters:
I20260808 17:45:48.002663 3859 leader_election.cc:288] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:48.002700 3871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:48.002868 3871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [term 1 LEADER]: Becoming Leader. State: Replica: 9ef44163d1d44cb8ae277f96f7b492ee, State: Running, Role: LEADER
I20260808 17:45:48.002936 3859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:48.003022 3871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ef44163d1d44cb8ae277f96f7b492ee" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40933 } }
I20260808 17:45:48.004017 3873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ef44163d1d44cb8ae277f96f7b492ee. Latest consensus state: current_term: 1 leader_uuid: "9ef44163d1d44cb8ae277f96f7b492ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ef44163d1d44cb8ae277f96f7b492ee" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40933 } } }
I20260808 17:45:48.004117 3873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:48.004336 3890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:48.004588 3872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ef44163d1d44cb8ae277f96f7b492ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ef44163d1d44cb8ae277f96f7b492ee" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40933 } } }
I20260808 17:45:48.004885 3890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:48.004928 3872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:48.006565 3890 catalog_manager.cc:1347] Generated new cluster ID: 33a592f1dcdc4799931c9984166df773
I20260808 17:45:48.006618 3890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:45:48 dist-test-slave-tk9z krb5kdc[3744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:48.013918 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.001991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.29.62.254:40933)
0808 17:45:48.002504 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.002513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.002546 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:45:48.002691 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.002699 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.002759 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.003000 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:48.003857 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.005320 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.006404 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.007010 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.007107 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.009992 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:48.009996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:48.009996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:48.010008 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:48.012146 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.012577 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.012580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.012580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.012690 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.012933 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.012934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.012934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.013214 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:48.013366 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.013485 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.013613 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":427,"threads_started":1}
I20260808 17:45:48.017876 3890 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:48.018456 3890 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:48.021919 3890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee: Generated new TSK 0
I20260808 17:45:48.022087 3890 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:48.049151 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40933
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:48.133149 3900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:48.133404 3900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:48.133446 3900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:48.134755 3900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:48.134825 3900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:48.134873 3900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:48.136521 3900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:48.136816 3900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:48.137071 3900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:48.142253 3913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:48.142261 3906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:48.142313 3900 server_base.cc:1034] running on GCE node
W20260808 17:45:48.142261 3907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:48.142733 3900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:48.142994 3900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:48.144161 3900 hybrid_clock.cc:648] HybridClock initialized: now 1786211148144155 us; error 35 us; skew 500 ppm
Aug 08 17:45:48 dist-test-slave-tk9z krb5kdc[3744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:48.146711 3900 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:48.147071 3900 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:48.147149 3900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:48.147307 3900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:48.148393 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "91de9514a1984bd6bec7647ffb2c26ee"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "28c3e7758052d61c77cdfde99ed57f7f"
server_key_iv: "e3e059523843cd9f3f6ce82797675d00"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.148793 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "91de9514a1984bd6bec7647ffb2c26ee"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "28c3e7758052d61c77cdfde99ed57f7f"
server_key_iv: "e3e059523843cd9f3f6ce82797675d00"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.150079 3900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 17:45:48.150897 3919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.151063 3900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.151175 3900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "91de9514a1984bd6bec7647ffb2c26ee"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "28c3e7758052d61c77cdfde99ed57f7f"
server_key_iv: "e3e059523843cd9f3f6ce82797675d00"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.151252 3900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:48.183971 3900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:48.186542 3900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:48.186724 3900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:48.186980 3900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:48.187314 3900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:48.187373 3900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.187424 3900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:48.187458 3900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.193225 3900 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:42401
I20260808 17:45:48.193325 4087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:42401 every 8 connection(s)
I20260808 17:45:48.193598 3900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:48.195300 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 3900
I20260808 17:45:48.195436 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:48.195647 29947 external_mini_cluster.cc:1428] Setting key 02e9cd5faa78fc365de7d7c3b4ff5555
Aug 08 17:45:48 dist-test-slave-tk9z krb5kdc[3744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:48.204470 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.194347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:59379 (local address 127.29.62.254:40933)
0808 17:45:48.194422 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.194424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.194640 (+ 216us) server_negotiation.cc:407] Connection header received
0808 17:45:48.195209 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.195210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.195224 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.195268 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:48.196674 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.197172 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.198624 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.198957 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.199032 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.201507 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:48.201509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:48.201510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:48.201517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:48.202956 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.203357 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.203359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.203359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.203422 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.203695 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.203696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.203696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.203817 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:48.203899 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.204285 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.204381 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 17:45:48.204646 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.194305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40933 (local address 127.29.62.193:59379)
0808 17:45:48.194518 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.194555 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:48.195076 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:48.195367 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:48.195373 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:48.195763 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:48.196577 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.196585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.197269 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.197270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.198535 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.198538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.199045 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.199046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.199170 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.199721 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:48.199731 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:48.201399 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:48.203036 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:48.203043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:48.203046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:48.203266 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:48.203507 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:48.203508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:48.203509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:48.203621 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:48.203890 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:48.203898 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:48.204138 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 17:45:48.204247 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 17:45:48.204410 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":117,"threads_started":1}
I20260808 17:45:48.205139 4088 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40933
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
I20260808 17:45:48.205235 4088 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:48.205399 4088 heartbeater.cc:511] Master 127.29.62.254:40933 requested a full tablet report, sending...
I20260808 17:45:48.205941 3804 ts_manager.cc:194] Registered new tserver with Master: 91de9514a1984bd6bec7647ffb2c26ee (127.29.62.193:42401)
I20260808 17:45:48.206704 3804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:59379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:48.222595 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40933
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:48.305421 4093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:48.305660 4093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:48.305701 4093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:48.306862 4093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:48.306933 4093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:48.306977 4093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:48.308471 4093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:48.308764 4093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:48.309000 4093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:48.314049 4101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:48.314098 4099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:48.314142 4093 server_base.cc:1034] running on GCE node
W20260808 17:45:48.314050 4098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:48.314461 4093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:48.314714 4093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:48.315889 4093 hybrid_clock.cc:648] HybridClock initialized: now 1786211148315877 us; error 25 us; skew 500 ppm
Aug 08 17:45:48 dist-test-slave-tk9z krb5kdc[3744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:48.318361 4093 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:48.318730 4093 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:48.318774 4093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:48.318862 4093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:48.319854 4093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "673ced66c44c474e94f56f2d6cef51dc"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "42f948b8170e3511f7112d8169bd49b9"
server_key_iv: "4053e6054ca6e8432033037697f16972"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.320163 4093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "673ced66c44c474e94f56f2d6cef51dc"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "42f948b8170e3511f7112d8169bd49b9"
server_key_iv: "4053e6054ca6e8432033037697f16972"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.321385 4093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:48.322057 4108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.322214 4093 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:48.322307 4093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "673ced66c44c474e94f56f2d6cef51dc"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "42f948b8170e3511f7112d8169bd49b9"
server_key_iv: "4053e6054ca6e8432033037697f16972"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.322363 4093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:48.329758 4093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:48.332060 4093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:48.332172 4093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:48.332348 4093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:48.332687 4093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:48.332726 4093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.332747 4093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:48.332759 4093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.338570 4093 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:33429
I20260808 17:45:48.338649 4289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:33429 every 8 connection(s)
I20260808 17:45:48.338914 4093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:45:48 dist-test-slave-tk9z krb5kdc[3744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:48.347694 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 4093
I20260808 17:45:48.347806 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:48.347903 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.339673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:37191 (local address 127.29.62.254:40933)
0808 17:45:48.339747 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.339749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.339919 (+ 170us) server_negotiation.cc:407] Connection header received
0808 17:45:48.340400 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.340401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.340410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.340463 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:48.341353 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.341842 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.342881 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.343074 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.343133 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.345164 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:48.345165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:48.345165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:48.345172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:48.346678 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.347034 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.347035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.347035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.347088 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.347300 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.347301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.347301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.347400 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:48.347450 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.347703 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.347792 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:45:48.348023 29947 external_mini_cluster.cc:1428] Setting key 68d362923d241f3bdd3b07ab43976393
I20260808 17:45:48.348121 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.339632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40933 (local address 127.29.62.194:37191)
0808 17:45:48.339794 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.339835 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:48.340317 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:48.340523 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:48.340526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:48.340779 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:48.341271 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.341276 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.341935 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.341936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.342805 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.342808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.343152 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.343153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.343254 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.343719 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:48.343728 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:48.345081 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:48.346748 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:48.346753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:48.346756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:48.346968 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:48.347149 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:48.347149 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:48.347150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:48.347244 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:48.347457 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:48.347462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:48.347571 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 17:45:48.347652 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:45:48.347804 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260808 17:45:48.348608 4291 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40933
I20260808 17:45:48.348698 4291 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:48.348896 4291 heartbeater.cc:511] Master 127.29.62.254:40933 requested a full tablet report, sending...
I20260808 17:45:48.349308 3803 ts_manager.cc:194] Registered new tserver with Master: 673ced66c44c474e94f56f2d6cef51dc (127.29.62.194:33429)
I20260808 17:45:48.349694 3803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:37191
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:48.373553 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40933
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:48.456646 4298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:48.456899 4298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:48.456943 4298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:48.458137 4298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:48.458213 4298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:48.458257 4298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:48.459815 4298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:48.460103 4298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:48.460351 4298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:48.465387 4308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:48.465425 4304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:48.465606 4298 server_base.cc:1034] running on GCE node
W20260808 17:45:48.465746 4305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:48.465961 4298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:48.466230 4298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:48.467407 4298 hybrid_clock.cc:648] HybridClock initialized: now 1786211148467400 us; error 50 us; skew 500 ppm
Aug 08 17:45:48 dist-test-slave-tk9z krb5kdc[3744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:48.470065 4298 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:48.470551 4298 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:48.470628 4298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:48.470793 4298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:48.471881 4298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "1955ef7832594dc587de5dec2a272c8f"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "6026870978b06d59b22185aa6857be39"
server_key_iv: "fde6b7417d8a1e104bce699604316395"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.472265 4298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "1955ef7832594dc587de5dec2a272c8f"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "6026870978b06d59b22185aa6857be39"
server_key_iv: "fde6b7417d8a1e104bce699604316395"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.473655 4298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:48.474498 4320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.474674 4298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:48.474792 4298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "1955ef7832594dc587de5dec2a272c8f"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "6026870978b06d59b22185aa6857be39"
server_key_iv: "fde6b7417d8a1e104bce699604316395"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.474875 4298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:48.491060 4298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:48.493033 4298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:48.493186 4298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:48.493458 4298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:48.493796 4298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:48.493830 4298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.493875 4298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:48.493906 4298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.500327 4298 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:33785
I20260808 17:45:48.500385 4481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:33785 every 8 connection(s)
I20260808 17:45:48.500650 4298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:45:48 dist-test-slave-tk9z krb5kdc[3744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211148, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:48.509002 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 4298
I20260808 17:45:48.509099 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:48.509348 29947 external_mini_cluster.cc:1428] Setting key 4a0cad23529a4773980baf80427d9413
I20260808 17:45:48.510345 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.501387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:51699 (local address 127.29.62.254:40933)
0808 17:45:48.501481 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.501483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.501680 (+ 197us) server_negotiation.cc:407] Connection header received
0808 17:45:48.502243 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.502245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.502258 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.502363 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:48.503438 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.504084 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.505217 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.505406 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.505462 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.507569 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:48.507571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:48.507571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:48.507577 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:48.508777 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.509258 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.509259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.509259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.509360 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.509654 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.509655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.509656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.509782 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:48.509868 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.510145 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.510232 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:45:48.510558 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.501367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40933 (local address 127.29.62.195:51699)
0808 17:45:48.501542 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.501589 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:48.502122 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:48.502497 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:48.502501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:48.502842 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:48.503329 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.503334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.504193 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.504194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.505141 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.505144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.505492 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.505493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.505590 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.506045 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:48.506053 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:48.507472 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:48.508887 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:48.508893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:48.508894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:48.509185 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:48.509444 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:48.509445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:48.509445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:48.509562 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:48.509858 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:48.509865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:48.510002 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 17:45:48.510119 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 17:45:48.510300 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260808 17:45:48.511103 4482 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40933
I20260808 17:45:48.511214 4482 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:48.511396 4482 heartbeater.cc:511] Master 127.29.62.254:40933 requested a full tablet report, sending...
I20260808 17:45:48.511765 3803 ts_manager.cc:194] Registered new tserver with Master: 1955ef7832594dc587de5dec2a272c8f (127.29.62.195:33785)
I20260808 17:45:48.512149 3803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:51699
I20260808 17:45:48.520996 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:45:48.528334 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.522476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.29.62.254:40933)
0808 17:45:48.522545 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.522547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.522668 (+ 121us) server_negotiation.cc:407] Connection header received
0808 17:45:48.522711 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.522712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.522721 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.522773 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:48.523331 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.523812 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.524523 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.524697 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.524747 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.525963 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:48.525964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:48.525964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:48.525971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:48.527313 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.527621 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.527622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.527622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.527675 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:48.527844 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:48.527844 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:48.527845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:48.527982 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:48.528057 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.528160 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.528221 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:45:48.530059 3803 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:48.531788 3803 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:48.538933 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.534619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42401 (local address 127.0.0.1:51304)
0808 17:45:48.534838 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.534851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:48.534918 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:48.535591 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:48.535592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:48.535595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:48.535640 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.535641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.536943 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.536944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.537838 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.537840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.538598 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.538600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.538679 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.538692 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:45:48.538795 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 17:45:48.538850 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
I20260808 17:45:48.539103 4499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.534543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33429 (local address 127.0.0.1:49926)
0808 17:45:48.534760 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.534805 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:48.534875 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:48.535229 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:48.535233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:48.535236 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:48.535310 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.535312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.536761 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.536762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.537863 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.537866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.538848 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.538849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.538923 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.538927 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:48.538958 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 17:45:48.539018 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260808 17:45:48.539103 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.534761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:33785 (local address 127.0.0.1:38386)
0808 17:45:48.535218 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.535228 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:48.535312 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:48.535510 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:48.535511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:48.535513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:48.535578 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.535580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.536540 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.536542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.537863 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.537866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.538899 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.538900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.538951 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.538954 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:48.539001 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:45:48.539030 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":16,"thread_start_us":91,"threads_started":1}
I20260808 17:45:48.539399 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.534673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.29.62.194:33429)
0808 17:45:48.535054 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.535058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.535067 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:48.535100 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.535102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.535115 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.535210 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:48.535399 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.536644 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.537971 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.538747 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.539146 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.539216 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.539244 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.539294 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":71,"threads_started":1}
I20260808 17:45:48.539403 4503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.534802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.29.62.193:42401)
0808 17:45:48.535365 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.535370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.535379 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:48.535416 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.535418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.535433 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.535716 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:48.535803 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.536830 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.537927 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.538501 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.538595 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.538658 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.538813 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.538853 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":107,"threads_started":1}
I20260808 17:45:48.539559 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.534982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.29.62.195:33785)
0808 17:45:48.535175 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.535179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.535309 (+ 130us) server_negotiation.cc:407] Connection header received
0808 17:45:48.535385 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.535387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.535410 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.535510 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:48.535665 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.536427 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.537972 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.538742 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.538818 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.538872 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.539417 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.539467 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":115,"threads_started":1}
I20260808 17:45:48.540199 4185 tablet_service.cc:1467] Processing CreateTablet for tablet 6e6b61e163a64a00bc2bee0910654985 (DEFAULT_TABLE table=test-table [id=11b5649656ba4c1aa426af8044e14ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:48.540199 3984 tablet_service.cc:1467] Processing CreateTablet for tablet 6e6b61e163a64a00bc2bee0910654985 (DEFAULT_TABLE table=test-table [id=11b5649656ba4c1aa426af8044e14ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:48.540323 4393 tablet_service.cc:1467] Processing CreateTablet for tablet 6e6b61e163a64a00bc2bee0910654985 (DEFAULT_TABLE table=test-table [id=11b5649656ba4c1aa426af8044e14ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:48.540589 4185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e6b61e163a64a00bc2bee0910654985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:48.540589 3984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e6b61e163a64a00bc2bee0910654985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:48.540728 4393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e6b61e163a64a00bc2bee0910654985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:48.543507 4508 tablet_bootstrap.cc:492] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: Bootstrap starting.
I20260808 17:45:48.543963 4510 tablet_bootstrap.cc:492] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Bootstrap starting.
I20260808 17:45:48.544335 4508 tablet_bootstrap.cc:654] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:48.544612 4510 tablet_bootstrap.cc:654] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:48.545022 4508 log.cc:824] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:48.545279 4510 log.cc:824] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:48.545670 4513 tablet_bootstrap.cc:492] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee: Bootstrap starting.
I20260808 17:45:48.545886 4508 tablet_bootstrap.cc:492] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: No bootstrap required, opened a new log
I20260808 17:45:48.545979 4508 ts_tablet_manager.cc:1397] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:45:48.546339 4513 tablet_bootstrap.cc:654] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:48.546917 4513 log.cc:824] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:48.547462 4510 tablet_bootstrap.cc:492] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: No bootstrap required, opened a new log
I20260808 17:45:48.547549 4510 ts_tablet_manager.cc:1397] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 17:45:48.547699 4513 tablet_bootstrap.cc:492] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee: No bootstrap required, opened a new log
I20260808 17:45:48.547776 4513 ts_tablet_manager.cc:1397] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:45:48.547900 4508 raft_consensus.cc:348] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.548081 4508 raft_consensus.cc:374] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:48.548130 4508 raft_consensus.cc:729] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1955ef7832594dc587de5dec2a272c8f, State: Initialized, Role: FOLLOWER
I20260808 17:45:48.548241 4508 consensus_queue.cc:260] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.548580 4508 ts_tablet_manager.cc:1428] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:45:48.548590 4482 heartbeater.cc:503] Master 127.29.62.254:40933 was elected leader, sending a full tablet report...
I20260808 17:45:48.549525 4510 raft_consensus.cc:348] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.549706 4510 raft_consensus.cc:374] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:48.549754 4510 raft_consensus.cc:729] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 673ced66c44c474e94f56f2d6cef51dc, State: Initialized, Role: FOLLOWER
I20260808 17:45:48.549866 4510 consensus_queue.cc:260] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.549916 4513 raft_consensus.cc:348] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.550071 4513 raft_consensus.cc:374] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:48.550117 4513 raft_consensus.cc:729] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91de9514a1984bd6bec7647ffb2c26ee, State: Initialized, Role: FOLLOWER
I20260808 17:45:48.550163 4510 ts_tablet_manager.cc:1428] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:45:48.550163 4291 heartbeater.cc:503] Master 127.29.62.254:40933 was elected leader, sending a full tablet report...
I20260808 17:45:48.550233 4513 consensus_queue.cc:260] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.550562 4513 ts_tablet_manager.cc:1428] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 17:45:48.550575 4088 heartbeater.cc:503] Master 127.29.62.254:40933 was elected leader, sending a full tablet report...
W20260808 17:45:48.589538 4292 tablet.cc:2369] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:45:48.668520 4522 raft_consensus.cc:482] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:48.668694 4522 raft_consensus.cc:504] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.669106 4522 leader_election.cc:288] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 673ced66c44c474e94f56f2d6cef51dc (127.29.62.194:33429), 91de9514a1984bd6bec7647ffb2c26ee (127.29.62.193:42401)
I20260808 17:45:48.672155 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.669302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42401 (local address 127.29.62.195:39391)
0808 17:45:48.669362 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.669375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:48.669442 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:48.669652 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:48.669654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:48.669656 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:48.669736 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.669738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.670524 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.670525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.671311 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.671314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.671856 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.671857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.671955 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.671957 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:45:48.672002 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:45:48.672032 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":18}
I20260808 17:45:48.672159 4503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.669465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:39391 (local address 127.29.62.193:42401)
0808 17:45:48.669516 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.669518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.669530 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:45:48.669565 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.669566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.669577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.669638 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:48.669833 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.670362 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.671414 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.671775 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.671832 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.671839 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.672032 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.672082 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:45:48.672209 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.669502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:49515 (local address 127.29.62.194:33429)
0808 17:45:48.669549 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.669551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.669584 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:45:48.669670 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.669671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.669680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.669739 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:48.669929 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.670454 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.671424 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.671796 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.671889 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.671897 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.672091 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.672134 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:45:48.672227 4526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.669334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33429 (local address 127.29.62.195:49515)
0808 17:45:48.669520 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.669530 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:48.669599 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:48.669754 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:48.669755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:48.669757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:48.669855 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.669857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.670551 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.670552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.671316 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:48.671318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.671914 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.671915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:48.671982 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.672006 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 17:45:48.672067 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 17:45:48.672101 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":101,"threads_started":1}
I20260808 17:45:48.672513 4217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e6b61e163a64a00bc2bee0910654985" candidate_uuid: "1955ef7832594dc587de5dec2a272c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "673ced66c44c474e94f56f2d6cef51dc" is_pre_election: true
I20260808 17:45:48.672560 4015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e6b61e163a64a00bc2bee0910654985" candidate_uuid: "1955ef7832594dc587de5dec2a272c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91de9514a1984bd6bec7647ffb2c26ee" is_pre_election: true
I20260808 17:45:48.672681 4217 raft_consensus.cc:2454] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1955ef7832594dc587de5dec2a272c8f in term 0.
I20260808 17:45:48.672716 4015 raft_consensus.cc:2454] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1955ef7832594dc587de5dec2a272c8f in term 0.
I20260808 17:45:48.672924 4321 leader_election.cc:302] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1955ef7832594dc587de5dec2a272c8f, 91de9514a1984bd6bec7647ffb2c26ee; no voters:
I20260808 17:45:48.673066 4522 raft_consensus.cc:2781] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:48.673125 4522 raft_consensus.cc:482] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:48.673177 4522 raft_consensus.cc:3037] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:48.673991 4522 raft_consensus.cc:504] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.674181 4522 leader_election.cc:288] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [CANDIDATE]: Term 1 election: Requested vote from peers 673ced66c44c474e94f56f2d6cef51dc (127.29.62.194:33429), 91de9514a1984bd6bec7647ffb2c26ee (127.29.62.193:42401)
I20260808 17:45:48.674320 4015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e6b61e163a64a00bc2bee0910654985" candidate_uuid: "1955ef7832594dc587de5dec2a272c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91de9514a1984bd6bec7647ffb2c26ee"
I20260808 17:45:48.674347 4217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e6b61e163a64a00bc2bee0910654985" candidate_uuid: "1955ef7832594dc587de5dec2a272c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "673ced66c44c474e94f56f2d6cef51dc"
I20260808 17:45:48.674444 4217 raft_consensus.cc:3037] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:48.674449 4015 raft_consensus.cc:3037] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:48.675205 4217 raft_consensus.cc:2454] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1955ef7832594dc587de5dec2a272c8f in term 1.
I20260808 17:45:48.675282 4015 raft_consensus.cc:2454] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1955ef7832594dc587de5dec2a272c8f in term 1.
I20260808 17:45:48.675424 4322 leader_election.cc:302] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1955ef7832594dc587de5dec2a272c8f, 673ced66c44c474e94f56f2d6cef51dc; no voters:
I20260808 17:45:48.675519 4522 raft_consensus.cc:2781] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:48.675668 4522 raft_consensus.cc:686] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 1 LEADER]: Becoming Leader. State: Replica: 1955ef7832594dc587de5dec2a272c8f, State: Running, Role: LEADER
I20260808 17:45:48.675789 4522 consensus_queue.cc:237] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } }
I20260808 17:45:48.676708 3803 catalog_manager.cc:5495] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1955ef7832594dc587de5dec2a272c8f (127.29.62.195). New cstate: current_term: 1 leader_uuid: "1955ef7832594dc587de5dec2a272c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1955ef7832594dc587de5dec2a272c8f" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 } health_report { overall_health: UNKNOWN } } }
I20260808 17:45:48.683109 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.680864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.29.62.195:33785)
0808 17:45:48.680920 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.680921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.680933 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:45:48.681017 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.681018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.681032 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.681090 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:48.681217 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.681665 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.682355 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:48.682591 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:48.682645 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:48.682781 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:48.682873 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:48.682931 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.682970 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.682990 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:45:48.684825 4015 raft_consensus.cc:1261] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 1 FOLLOWER]: Refusing update from remote peer 1955ef7832594dc587de5dec2a272c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:48.684847 4217 raft_consensus.cc:1261] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 1 FOLLOWER]: Refusing update from remote peer 1955ef7832594dc587de5dec2a272c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:48.685215 4527 consensus_queue.cc:1035] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "91de9514a1984bd6bec7647ffb2c26ee" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:48.685345 4522 consensus_queue.cc:1035] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "673ced66c44c474e94f56f2d6cef51dc" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:48.687589 4536 mvcc.cc:205] Tried to move back new op lower bound from 7316320865010688000 to 7316320864976965632. Current Snapshot: MvccSnapshot[applied={T|T < 7316320865010688000}]
I20260808 17:45:48.688213 4537 mvcc.cc:205] Tried to move back new op lower bound from 7316320865010688000 to 7316320864976965632. Current Snapshot: MvccSnapshot[applied={T|T < 7316320865010688000}]
I20260808 17:45:48.690526 3804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:48.690631 3804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:48.691363 3804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ef44163d1d44cb8ae277f96f7b492ee: Sending DeleteTablet for 3 replicas of tablet 6e6b61e163a64a00bc2bee0910654985
I20260808 17:45:48.691677 4185 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e6b61e163a64a00bc2bee0910654985 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:48 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:49926
I20260808 17:45:48.691725 3984 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e6b61e163a64a00bc2bee0910654985 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:48 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:51304
I20260808 17:45:48.691895 4393 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e6b61e163a64a00bc2bee0910654985 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:48 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:38386
I20260808 17:45:48.691936 4550 tablet_replica.cc:330] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee: stopping tablet replica
I20260808 17:45:48.692039 4550 raft_consensus.cc:2229] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:48.692139 4550 raft_consensus.cc:2258] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:48.692337 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 3900
I20260808 17:45:48.692767 4550 ts_tablet_manager.cc:1905] T 6e6b61e163a64a00bc2bee0910654985 P 91de9514a1984bd6bec7647ffb2c26ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:48.697449 4549 tablet_replica.cc:330] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: stopping tablet replica
I20260808 17:45:48.697573 4549 raft_consensus.cc:2229] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:48.697676 4549 raft_consensus.cc:2258] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:48.697842 4551 tablet_replica.cc:330] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: stopping tablet replica
I20260808 17:45:48.697934 4551 raft_consensus.cc:2229] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:48.698065 4551 raft_consensus.cc:2258] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:48.698469 4549 ts_tablet_manager.cc:1905] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:48.698912 4551 ts_tablet_manager.cc:1905] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:48.700317 4549 ts_tablet_manager.cc:1918] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:48.700448 4549 log.cc:1196] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/6e6b61e163a64a00bc2bee0910654985
I20260808 17:45:48.700613 4551 ts_tablet_manager.cc:1918] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:48.700691 4551 log.cc:1196] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/6e6b61e163a64a00bc2bee0910654985
I20260808 17:45:48.700924 4549 ts_tablet_manager.cc:1939] T 6e6b61e163a64a00bc2bee0910654985 P 673ced66c44c474e94f56f2d6cef51dc: Deleting consensus metadata
I20260808 17:45:48.700943 4551 ts_tablet_manager.cc:1939] T 6e6b61e163a64a00bc2bee0910654985 P 1955ef7832594dc587de5dec2a272c8f: Deleting consensus metadata
I20260808 17:45:48.701337 3784 catalog_manager.cc:4841] TS 1955ef7832594dc587de5dec2a272c8f (127.29.62.195:33785): tablet 6e6b61e163a64a00bc2bee0910654985 (table test-table [id=11b5649656ba4c1aa426af8044e14ca3]) successfully deleted
I20260808 17:45:48.701520 3784 catalog_manager.cc:4841] TS 673ced66c44c474e94f56f2d6cef51dc (127.29.62.194:33429): tablet 6e6b61e163a64a00bc2bee0910654985 (table test-table [id=11b5649656ba4c1aa426af8044e14ca3]) successfully deleted
W20260808 17:45:48.701828 3783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:42401 (error 108)
I20260808 17:45:48.702199 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 4093
I20260808 17:45:48.702302 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.702168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42401 (local address 127.0.0.1:51314)
0808 17:45:48.702226 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:48.702265 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:42401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 17:45:48.704785 3783 catalog_manager.cc:4584] TS 91de9514a1984bd6bec7647ffb2c26ee (127.29.62.193:42401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e6b61e163a64a00bc2bee0910654985: Network error: Client connection negotiation failed: client connection to 127.29.62.193:42401: connect: Connection refused (error 111)
I20260808 17:45:48.712857 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 4298
I20260808 17:45:48.725615 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 3764
2026-08-08T17:45:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2218 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T17:45:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:48Z Disabled control of system clock
I20260808 17:45:48.759256 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40399
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:37823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 17:45:48.846308 4557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:48.846539 4557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:48.846561 4557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:48.847633 4557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:48.847666 4557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:48.847678 4557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:48.847692 4557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:48.847703 4557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:48.848980 4557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:48.849172 4557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:48.849411 4557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:48.854228 4567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:48.854295 4564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:48.854414 4557 server_base.cc:1034] running on GCE node
W20260808 17:45:48.854420 4565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:48.854874 4557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:48.855212 4557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:48.856438 4557 hybrid_clock.cc:648] HybridClock initialized: now 1786211148856422 us; error 27 us; skew 500 ppm
I20260808 17:45:48.857858 4557 webserver.cc:460] Webserver started at http://127.29.62.254:46529/ using document root <none> and password file <none>
I20260808 17:45:48.858161 4557 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:48.858245 4557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:48.858484 4557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:48.859673 4557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "8e9bbb381ec248ada521b866e1ffd7c2"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "ccb4012f9251127c07cafb10dd9ef2d5"
server_key_iv: "3f63ae317df21d9da6ebca6b5346c317"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.860095 4557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "8e9bbb381ec248ada521b866e1ffd7c2"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "ccb4012f9251127c07cafb10dd9ef2d5"
server_key_iv: "3f63ae317df21d9da6ebca6b5346c317"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.861454 4557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:48.862205 4576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:48.862442 4557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:48.862557 4557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "8e9bbb381ec248ada521b866e1ffd7c2"
format_stamp: "Formatted at 2026-08-08 17:45:48 on dist-test-slave-tk9z"
server_key: "ccb4012f9251127c07cafb10dd9ef2d5"
server_key_iv: "3f63ae317df21d9da6ebca6b5346c317"
server_key_version: "encryptionkey@0"
I20260808 17:45:48.862651 4557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:48.866307 4557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:48.866616 4557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:48.866792 4557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:48.871163 4557 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40399
I20260808 17:45:48.871191 4658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40399 every 8 connection(s)
I20260808 17:45:48.871580 4557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:48.872313 4659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:48.873844 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 4557
I20260808 17:45:48.873981 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:48.874181 29947 external_mini_cluster.cc:1428] Setting key e69e2b05b87b38562de0d13af7b4d8ff
I20260808 17:45:48.875633 4659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2: Bootstrap starting.
I20260808 17:45:48.876334 4659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:48.876698 4659 log.cc:824] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:48.877005 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:48.874791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54452 (local address 127.29.62.254:40399)
0808 17:45:48.875011 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:48.875019 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:48.875052 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:45:48.875574 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:48.875583 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:48.875637 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:48.875789 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:48.876532 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:48.876536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:48.876537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:48.876542 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:48.876592 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:48.876608 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:48.876686 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:48.876751 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:45:48.876808 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260808 17:45:48.877519 4659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2: No bootstrap required, opened a new log
I20260808 17:45:48.878958 4659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9bbb381ec248ada521b866e1ffd7c2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40399 } }
I20260808 17:45:48.879098 4659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:48.879120 4659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9bbb381ec248ada521b866e1ffd7c2, State: Initialized, Role: FOLLOWER
I20260808 17:45:48.879297 4659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9bbb381ec248ada521b866e1ffd7c2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40399 } }
I20260808 17:45:48.879386 4659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:48.879431 4659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:48.879480 4659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:48.880249 4659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9bbb381ec248ada521b866e1ffd7c2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40399 } }
I20260808 17:45:48.880338 4659 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9bbb381ec248ada521b866e1ffd7c2; no voters:
I20260808 17:45:48.880546 4659 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:48.880604 4671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:48.880786 4671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9bbb381ec248ada521b866e1ffd7c2, State: Running, Role: LEADER
I20260808 17:45:48.880846 4659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:48.880914 4671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9bbb381ec248ada521b866e1ffd7c2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40399 } }
I20260808 17:45:48.881382 4673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9bbb381ec248ada521b866e1ffd7c2. Latest consensus state: current_term: 1 leader_uuid: "8e9bbb381ec248ada521b866e1ffd7c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9bbb381ec248ada521b866e1ffd7c2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40399 } } }
I20260808 17:45:48.881502 4673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:48.881369 4672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e9bbb381ec248ada521b866e1ffd7c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9bbb381ec248ada521b866e1ffd7c2" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40399 } } }
I20260808 17:45:48.881717 4672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:48.881737 4684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:48.882442 4684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:48.884222 4684 catalog_manager.cc:1347] Generated new cluster ID: 50c3bf0ed76c46b385744c3f4fc645b1
I20260808 17:45:48.884275 4684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:48.887703 4684 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:48.887868 4684 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:48.897603 4684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2: Generated new TSK 0
I20260808 17:45:48.897781 4684 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:45:48.904042 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40399
--builtin_ntp_servers=127.29.62.212:37823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 17:45:48.994069 4700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:48.994266 4700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:48.994288 4700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:48.995486 4700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:48.995528 4700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:48.995554 4700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:48.996965 4700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:48.997243 4700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:48.997581 4700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:49.002727 4714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:49.002741 4708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:49.002956 4700 server_base.cc:1034] running on GCE node
W20260808 17:45:49.003036 4707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:49.003254 4700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:49.003575 4700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:49.004746 4700 hybrid_clock.cc:648] HybridClock initialized: now 1786211149004733 us; error 34 us; skew 500 ppm
I20260808 17:45:49.006071 4700 webserver.cc:460] Webserver started at http://127.29.62.193:33879/ using document root <none> and password file <none>
I20260808 17:45:49.006412 4700 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:49.006476 4700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:49.006631 4700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:49.007706 4700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "6ef0338866b6482fa844dff54d545eea"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "68e96fafabd12719fbcc4f504fb0d191"
server_key_iv: "5bb8e44d4136f5c04389f27feb54d009"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.008077 4700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "6ef0338866b6482fa844dff54d545eea"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "68e96fafabd12719fbcc4f504fb0d191"
server_key_iv: "5bb8e44d4136f5c04389f27feb54d009"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.009456 4700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:49.010233 4721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.010452 4700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:49.010562 4700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "6ef0338866b6482fa844dff54d545eea"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "68e96fafabd12719fbcc4f504fb0d191"
server_key_iv: "5bb8e44d4136f5c04389f27feb54d009"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.010640 4700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:49.013993 4700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:49.014254 4700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:49.014454 4700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:49.014678 4700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:49.014988 4700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:49.015023 4700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.015043 4700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:49.015093 4700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.021010 4700 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:36669
I20260808 17:45:49.021061 4892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:36669 every 8 connection(s)
I20260808 17:45:49.021386 4700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:49.024044 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 4700
I20260808 17:45:49.024178 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:49.024364 29947 external_mini_cluster.cc:1428] Setting key 42c3458581fb0d33d1e6657a659afbbb
I20260808 17:45:49.024399 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.022057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:35661 (local address 127.29.62.254:40399)
0808 17:45:49.022138 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.022140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.022430 (+ 290us) server_negotiation.cc:407] Connection header received
0808 17:45:49.022938 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.022939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.022945 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.023002 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.024103 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.024105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.024105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.024110 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.024125 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.024130 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.024189 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.024303 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.024329 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 17:45:49.024595 4899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.022047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40399 (local address 127.29.62.193:35661)
0808 17:45:49.022273 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.022305 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.022846 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.023053 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.023056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.023065 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:49.023942 (+ 877us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:45:49.023943 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:49.023971 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:45:49.024001 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:49.024203 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:49.024209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:49.024212 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:49.024311 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 17:45:49.024375 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":132,"threads_started":1}
I20260808 17:45:49.025079 4893 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40399
I20260808 17:45:49.025205 4893 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:49.025354 4893 heartbeater.cc:511] Master 127.29.62.254:40399 requested a full tablet report, sending...
I20260808 17:45:49.025480 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40399
--builtin_ntp_servers=127.29.62.212:37823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 17:45:49.025919 4600 ts_manager.cc:194] Registered new tserver with Master: 6ef0338866b6482fa844dff54d545eea (127.29.62.193:36669)
W20260808 17:45:49.109654 4900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:49.109848 4900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:49.109870 4900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:49.111240 4900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:49.111294 4900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:49.111377 4900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:49.112916 4900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:49.113225 4900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:49.113554 4900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:49.118621 4911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:49.118678 4909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:49.118675 4907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:49.118850 4900 server_base.cc:1034] running on GCE node
I20260808 17:45:49.119026 4900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:49.119210 4900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:49.120393 4900 hybrid_clock.cc:648] HybridClock initialized: now 1786211149120386 us; error 36 us; skew 500 ppm
I20260808 17:45:49.121757 4900 webserver.cc:460] Webserver started at http://127.29.62.194:43889/ using document root <none> and password file <none>
I20260808 17:45:49.122043 4900 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:49.122090 4900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:49.122269 4900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:49.123368 4900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "92efe2a86f9e445a81fa1e9cce89b9d2"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "7ad98f8b0edb3859ef407292ea03fa12"
server_key_iv: "7192aac8a1d51508b664a2dac6031a27"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.123734 4900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "92efe2a86f9e445a81fa1e9cce89b9d2"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "7ad98f8b0edb3859ef407292ea03fa12"
server_key_iv: "7192aac8a1d51508b664a2dac6031a27"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.125119 4900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:49.125831 4918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.126039 4900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:49.126128 4900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "92efe2a86f9e445a81fa1e9cce89b9d2"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "7ad98f8b0edb3859ef407292ea03fa12"
server_key_iv: "7192aac8a1d51508b664a2dac6031a27"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.126186 4900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:49.129449 4900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:49.129715 4900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:49.129858 4900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:49.130070 4900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:49.130442 4900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:49.130486 4900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.130535 4900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:49.130565 4900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.136211 4900 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:43761
I20260808 17:45:49.136296 5084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:43761 every 8 connection(s)
I20260808 17:45:49.136619 4900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:49.139755 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.137377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:53749 (local address 127.29.62.254:40399)
0808 17:45:49.137442 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.137444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.137705 (+ 261us) server_negotiation.cc:407] Connection header received
0808 17:45:49.138242 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.138243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.138250 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.138302 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.139465 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.139467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.139467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.139472 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.139490 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.139494 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.139570 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.139665 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.139681 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:45:49.139964 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.137350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40399 (local address 127.29.62.194:53749)
0808 17:45:49.137571 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.137616 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.138122 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.138393 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.138398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.138407 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:49.139296 (+ 889us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:45:49.139297 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:49.139326 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:45:49.139354 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:49.139569 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:49.139576 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:49.139580 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:49.139675 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 17:45:49.139736 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260808 17:45:49.140537 5085 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40399
I20260808 17:45:49.140633 5085 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:49.140753 5085 heartbeater.cc:511] Master 127.29.62.254:40399 requested a full tablet report, sending...
I20260808 17:45:49.141120 4600 ts_manager.cc:194] Registered new tserver with Master: 92efe2a86f9e445a81fa1e9cce89b9d2 (127.29.62.194:43761)
I20260808 17:45:49.142145 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 4900
I20260808 17:45:49.142263 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:49.142519 29947 external_mini_cluster.cc:1428] Setting key 50f3a5a124f11273c56a58b8c029d038
I20260808 17:45:49.143437 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40399
--builtin_ntp_servers=127.29.62.212:37823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 17:45:49.227591 5092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:49.227815 5092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:49.227854 5092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:49.229054 5092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:49.229115 5092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:49.229158 5092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:49.230677 5092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:49.230940 5092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:49.231228 5092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:49.236217 5100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:49.236183 5105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:49.236234 5099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:49.236377 5092 server_base.cc:1034] running on GCE node
I20260808 17:45:49.236593 5092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:49.236835 5092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:49.238019 5092 hybrid_clock.cc:648] HybridClock initialized: now 1786211149238016 us; error 36 us; skew 500 ppm
I20260808 17:45:49.239332 5092 webserver.cc:460] Webserver started at http://127.29.62.195:38451/ using document root <none> and password file <none>
I20260808 17:45:49.239612 5092 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:49.239687 5092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:49.239850 5092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:49.240935 5092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "a4e58f118ffb2c235aed88019b46561e"
server_key_iv: "f1e73391a913860aa1d321f14d8de1d1"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.241319 5092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "a4e58f118ffb2c235aed88019b46561e"
server_key_iv: "f1e73391a913860aa1d321f14d8de1d1"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.242646 5092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:49.243341 5114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.243516 5092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.243628 5092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "a4e58f118ffb2c235aed88019b46561e"
server_key_iv: "f1e73391a913860aa1d321f14d8de1d1"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.243706 5092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:49.247099 5092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:49.247350 5092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:49.247495 5092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:49.247704 5092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:49.248013 5092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:49.248070 5092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.248109 5092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:49.248137 5092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.253973 5092 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:33705
I20260808 17:45:49.254015 5293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:33705 every 8 connection(s)
I20260808 17:45:49.254345 5092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:45:49.257200 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.255091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:37959 (local address 127.29.62.254:40399)
0808 17:45:49.255150 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.255152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.255379 (+ 227us) server_negotiation.cc:407] Connection header received
0808 17:45:49.255814 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.255815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.255820 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.255875 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.257023 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.257023 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.257023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.257026 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.257036 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.257039 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.257078 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.257137 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.257149 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:45:49.257370 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.255058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40399 (local address 127.29.62.195:37959)
0808 17:45:49.255254 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.255292 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.255731 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.255923 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.255926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.255934 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:49.256888 (+ 954us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:45:49.256888 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:49.256915 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:45:49.256944 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:49.257079 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:49.257081 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:49.257083 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:45:49.257132 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 17:45:49.257176 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260808 17:45:49.257851 5295 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40399
I20260808 17:45:49.257939 5295 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:49.258051 5295 heartbeater.cc:511] Master 127.29.62.254:40399 requested a full tablet report, sending...
I20260808 17:45:49.258446 4600 ts_manager.cc:194] Registered new tserver with Master: 7ae3990f57fb42e3b3e8ca11c1ddedb4 (127.29.62.195:33705)
I20260808 17:45:49.258829 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 5092
I20260808 17:45:49.258937 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:49.259112 29947 external_mini_cluster.cc:1428] Setting key 8ecfa53ba5d1060970c7a22bb16c7c34
I20260808 17:45:49.260021 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:45:49.262248 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.261263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54460 (local address 127.29.62.254:40399)
0808 17:45:49.261341 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.261344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.261442 (+ 98us) server_negotiation.cc:407] Connection header received
0808 17:45:49.261504 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.261504 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.261509 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.261558 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.262047 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.262048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.262049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.262054 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.262064 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.262068 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.262117 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.262170 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.262183 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 17:45:49.263721 4600 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:49.265239 4600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:49.269265 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.267604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:43761 (local address 127.0.0.1:33872)
0808 17:45:49.267962 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.267972 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.268005 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.268513 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.268514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.268517 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:49.268879 (+ 362us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:45:49.268880 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:49.268885 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:45:49.268896 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:49.269110 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:49.269111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:49.269112 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:45:49.269169 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 17:45:49.269182 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":70,"threads_started":1}
I20260808 17:45:49.269279 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.267840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33872 (local address 127.29.62.194:43761)
0808 17:45:49.268056 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.268061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.268072 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:45:49.268097 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.268099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.268108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.268180 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.268986 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.268994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.268994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.268999 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.269037 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.269058 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.269114 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.269178 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.269194 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
I20260808 17:45:49.269438 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.267391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:36669 (local address 127.0.0.1:55772)
0808 17:45:49.267738 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.267790 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.267860 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.268529 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.268530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.268532 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:49.269042 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:45:49.269043 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:49.269050 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:45:49.269063 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:49.269304 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:49.269304 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:49.269305 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:45:49.269350 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:45:49.269363 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":60,"threads_started":1}
I20260808 17:45:49.269474 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.267490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.29.62.195:33705)
0808 17:45:49.267680 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.267683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.267856 (+ 173us) server_negotiation.cc:407] Connection header received
0808 17:45:49.267909 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.267912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.267922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.267993 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.268926 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.268933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.268934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.268939 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.268976 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.268999 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.269061 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.269233 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.269249 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260808 17:45:49.269454 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.267397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:33705 (local address 127.0.0.1:44824)
0808 17:45:49.267749 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.267790 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.267857 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.268271 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.268277 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.268284 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:49.268777 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:45:49.268778 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:49.268790 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:45:49.268832 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:49.269060 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:49.269067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:49.269071 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:49.269137 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:45:49.269150 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":99,"threads_started":1}
I20260808 17:45:49.269673 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.267477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.29.62.193:36669)
0808 17:45:49.267638 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.267643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.268068 (+ 425us) server_negotiation.cc:407] Connection header received
0808 17:45:49.268101 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.268103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.268114 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.268187 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.269150 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.269157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.269157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.269161 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.269201 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.269224 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.269280 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.269554 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.269570 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260808 17:45:49.270143 4981 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffd232a4798432b9cbceacdc65b20a9 (DEFAULT_TABLE table=test-table [id=eaf1085aa0704425b05019dc8acb4784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:49.270227 4788 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffd232a4798432b9cbceacdc65b20a9 (DEFAULT_TABLE table=test-table [id=eaf1085aa0704425b05019dc8acb4784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:49.270319 5190 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffd232a4798432b9cbceacdc65b20a9 (DEFAULT_TABLE table=test-table [id=eaf1085aa0704425b05019dc8acb4784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:49.270581 4981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffd232a4798432b9cbceacdc65b20a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:49.270581 4788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffd232a4798432b9cbceacdc65b20a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:49.270749 5190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffd232a4798432b9cbceacdc65b20a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:49.273375 5313 tablet_bootstrap.cc:492] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: Bootstrap starting.
I20260808 17:45:49.273917 5314 tablet_bootstrap.cc:492] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Bootstrap starting.
I20260808 17:45:49.274015 5313 tablet_bootstrap.cc:654] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:49.274358 5313 log.cc:824] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:49.274359 5315 tablet_bootstrap.cc:492] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea: Bootstrap starting.
I20260808 17:45:49.274766 5314 tablet_bootstrap.cc:654] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:49.275063 5315 tablet_bootstrap.cc:654] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:49.275115 5314 log.cc:824] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:49.275204 5313 tablet_bootstrap.cc:492] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: No bootstrap required, opened a new log
I20260808 17:45:49.275269 5313 ts_tablet_manager.cc:1397] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:45:49.275398 5315 log.cc:824] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:49.276100 5315 tablet_bootstrap.cc:492] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea: No bootstrap required, opened a new log
I20260808 17:45:49.276155 5314 tablet_bootstrap.cc:492] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: No bootstrap required, opened a new log
I20260808 17:45:49.276187 5315 ts_tablet_manager.cc:1397] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:49.276224 5314 ts_tablet_manager.cc:1397] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:45:49.277944 5315 raft_consensus.cc:348] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.277968 5313 raft_consensus.cc:348] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.278107 5315 raft_consensus.cc:374] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:49.278149 5315 raft_consensus.cc:729] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef0338866b6482fa844dff54d545eea, State: Initialized, Role: FOLLOWER
I20260808 17:45:49.278151 5313 raft_consensus.cc:374] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:49.278194 5313 raft_consensus.cc:729] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae3990f57fb42e3b3e8ca11c1ddedb4, State: Initialized, Role: FOLLOWER
I20260808 17:45:49.278257 5315 consensus_queue.cc:260] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.278330 5313 consensus_queue.cc:260] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.278594 5315 ts_tablet_manager.cc:1428] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:45:49.278606 4893 heartbeater.cc:503] Master 127.29.62.254:40399 was elected leader, sending a full tablet report...
I20260808 17:45:49.278648 5313 ts_tablet_manager.cc:1428] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 17:45:49.278760 5295 heartbeater.cc:503] Master 127.29.62.254:40399 was elected leader, sending a full tablet report...
I20260808 17:45:49.279135 5314 raft_consensus.cc:348] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.279291 5314 raft_consensus.cc:374] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:49.279343 5314 raft_consensus.cc:729] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92efe2a86f9e445a81fa1e9cce89b9d2, State: Initialized, Role: FOLLOWER
I20260808 17:45:49.279516 5314 consensus_queue.cc:260] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.279798 5085 heartbeater.cc:503] Master 127.29.62.254:40399 was elected leader, sending a full tablet report...
I20260808 17:45:49.280025 5314 ts_tablet_manager.cc:1428] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
W20260808 17:45:49.387172 5086 tablet.cc:2369] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:45:49.428270 5324 raft_consensus.cc:482] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:49.428428 5324 raft_consensus.cc:504] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.428821 5324 leader_election.cc:288] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ef0338866b6482fa844dff54d545eea (127.29.62.193:36669), 7ae3990f57fb42e3b3e8ca11c1ddedb4 (127.29.62.195:33705)
I20260808 17:45:49.430405 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.429117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:58685 (local address 127.29.62.193:36669)
0808 17:45:49.429244 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.429245 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.429260 (+ 15us) server_negotiation.cc:407] Connection header received
0808 17:45:49.429285 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.429286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.429291 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.429375 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.430051 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.430053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.430054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.430059 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.430079 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.430084 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.430172 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.430276 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.430293 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 17:45:49.430418 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.429017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:36669 (local address 127.29.62.194:58685)
0808 17:45:49.429093 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.429105 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.429162 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.429384 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.429385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.429388 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:49.429909 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:45:49.429910 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:49.429919 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:45:49.429935 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:49.430182 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:49.430184 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:49.430185 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:45:49.430240 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:45:49.430302 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 17:45:49.430577 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.429058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:33705 (local address 127.29.62.194:48661)
0808 17:45:49.429245 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.429254 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.429335 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.429498 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.429499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.429501 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:49.430092 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:45:49.430093 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:49.430099 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:45:49.430112 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:49.430362 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:49.430363 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:49.430364 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:45:49.430470 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 17:45:49.430484 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":121,"threads_started":1}
I20260808 17:45:49.430603 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.429144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:48661 (local address 127.29.62.195:33705)
0808 17:45:49.429203 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.429205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.429332 (+ 127us) server_negotiation.cc:407] Connection header received
0808 17:45:49.429401 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.429402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.429407 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.429490 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.430236 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.430237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.430238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.430243 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.430275 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.430281 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.430316 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.430505 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.430523 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:45:49.430833 4826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffd232a4798432b9cbceacdc65b20a9" candidate_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef0338866b6482fa844dff54d545eea" is_pre_election: true
I20260808 17:45:49.430996 4826 raft_consensus.cc:2454] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92efe2a86f9e445a81fa1e9cce89b9d2 in term 0.
I20260808 17:45:49.430953 5221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffd232a4798432b9cbceacdc65b20a9" candidate_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" is_pre_election: true
I20260808 17:45:49.431118 5221 raft_consensus.cc:2454] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92efe2a86f9e445a81fa1e9cce89b9d2 in term 0.
I20260808 17:45:49.431282 4920 leader_election.cc:302] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ef0338866b6482fa844dff54d545eea, 92efe2a86f9e445a81fa1e9cce89b9d2; no voters:
I20260808 17:45:49.431429 5324 raft_consensus.cc:2781] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:49.431506 5324 raft_consensus.cc:482] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:49.431541 5324 raft_consensus.cc:3037] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:49.432284 5324 raft_consensus.cc:504] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.432448 5324 leader_election.cc:288] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [CANDIDATE]: Term 1 election: Requested vote from peers 6ef0338866b6482fa844dff54d545eea (127.29.62.193:36669), 7ae3990f57fb42e3b3e8ca11c1ddedb4 (127.29.62.195:33705)
I20260808 17:45:49.432615 4826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffd232a4798432b9cbceacdc65b20a9" candidate_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef0338866b6482fa844dff54d545eea"
I20260808 17:45:49.432647 5221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffd232a4798432b9cbceacdc65b20a9" candidate_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4"
I20260808 17:45:49.432701 4826 raft_consensus.cc:3037] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:49.432717 5221 raft_consensus.cc:3037] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:49.433614 4826 raft_consensus.cc:2454] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92efe2a86f9e445a81fa1e9cce89b9d2 in term 1.
I20260808 17:45:49.433614 5221 raft_consensus.cc:2454] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92efe2a86f9e445a81fa1e9cce89b9d2 in term 1.
I20260808 17:45:49.433805 4919 leader_election.cc:302] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ae3990f57fb42e3b3e8ca11c1ddedb4, 92efe2a86f9e445a81fa1e9cce89b9d2; no voters:
I20260808 17:45:49.433911 5324 raft_consensus.cc:2781] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:49.434064 5324 raft_consensus.cc:686] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 1 LEADER]: Becoming Leader. State: Replica: 92efe2a86f9e445a81fa1e9cce89b9d2, State: Running, Role: LEADER
I20260808 17:45:49.434193 5324 consensus_queue.cc:237] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } }
I20260808 17:45:49.435086 4599 catalog_manager.cc:5495] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92efe2a86f9e445a81fa1e9cce89b9d2 (127.29.62.194). New cstate: current_term: 1 leader_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92efe2a86f9e445a81fa1e9cce89b9d2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 43761 } health_report { overall_health: HEALTHY } } }
I20260808 17:45:49.451526 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.450542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33882 (local address 127.29.62.194:43761)
0808 17:45:49.450621 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.450623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.450631 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:45:49.450665 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.450666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.450678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.450728 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.451353 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.451354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.451354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.451357 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.451371 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.451375 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.451415 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.451453 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.451464 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 17:45:49.453313 4826 raft_consensus.cc:1261] T 1ffd232a4798432b9cbceacdc65b20a9 P 6ef0338866b6482fa844dff54d545eea [term 1 FOLLOWER]: Refusing update from remote peer 92efe2a86f9e445a81fa1e9cce89b9d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:49.453336 5221 raft_consensus.cc:1261] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 1 FOLLOWER]: Refusing update from remote peer 92efe2a86f9e445a81fa1e9cce89b9d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:49.453722 5329 consensus_queue.cc:1035] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae3990f57fb42e3b3e8ca11c1ddedb4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:49.453860 5324 consensus_queue.cc:1035] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ef0338866b6482fa844dff54d545eea" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 36669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:49.456558 5333 mvcc.cc:205] Tried to move back new op lower bound from 7316320868158197760 to 7316320868083044352. Current Snapshot: MvccSnapshot[applied={T|T < 7316320868158197760}]
I20260808 17:45:49.457335 5334 mvcc.cc:205] Tried to move back new op lower bound from 7316320868158197760 to 7316320868083044352. Current Snapshot: MvccSnapshot[applied={T|T < 7316320868158197760 or (T in {7316320868158197760})}]
I20260808 17:45:49.458833 4599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:49.458915 4599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:49.459692 4599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e9bbb381ec248ada521b866e1ffd7c2: Sending DeleteTablet for 3 replicas of tablet 1ffd232a4798432b9cbceacdc65b20a9
I20260808 17:45:49.460060 4788 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ffd232a4798432b9cbceacdc65b20a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:49 UTC) from {username='slave'} at 127.0.0.1:55772
I20260808 17:45:49.460062 5190 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ffd232a4798432b9cbceacdc65b20a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:49 UTC) from {username='slave'} at 127.0.0.1:44824
I20260808 17:45:49.460268 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 4700
I20260808 17:45:49.464233 4981 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ffd232a4798432b9cbceacdc65b20a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:49 UTC) from {username='slave'} at 127.0.0.1:33872
I20260808 17:45:49.474871 5351 tablet_replica.cc:330] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: stopping tablet replica
I20260808 17:45:49.475013 5351 raft_consensus.cc:2229] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:49.475078 5350 tablet_replica.cc:330] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: stopping tablet replica
I20260808 17:45:49.475172 5351 raft_consensus.cc:2258] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:49.475208 5350 raft_consensus.cc:2229] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:49.475330 5350 raft_consensus.cc:2258] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:49.475685 5351 ts_tablet_manager.cc:1905] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:49.477470 5351 ts_tablet_manager.cc:1918] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:49.477555 5351 log.cc:1196] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/1ffd232a4798432b9cbceacdc65b20a9
I20260808 17:45:49.477849 5351 ts_tablet_manager.cc:1939] T 1ffd232a4798432b9cbceacdc65b20a9 P 92efe2a86f9e445a81fa1e9cce89b9d2: Deleting consensus metadata
I20260808 17:45:49.478297 4578 catalog_manager.cc:4841] TS 92efe2a86f9e445a81fa1e9cce89b9d2 (127.29.62.194:43761): tablet 1ffd232a4798432b9cbceacdc65b20a9 (table test-table [id=eaf1085aa0704425b05019dc8acb4784]) successfully deleted
W20260808 17:45:49.479655 4578 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:36669 (error 108)
I20260808 17:45:49.480302 5350 ts_tablet_manager.cc:1905] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 17:45:49.482089 5311 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:49.480047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:36669 (local address 127.0.0.1:55784)
0808 17:45:49.480118 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.480132 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.480180 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.482026 (+ 1846us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:36669: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 17:45:49.482096 5350 ts_tablet_manager.cc:1918] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:49.482167 5350 log.cc:1196] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/1ffd232a4798432b9cbceacdc65b20a9
W20260808 17:45:49.482281 4578 catalog_manager.cc:4584] TS 6ef0338866b6482fa844dff54d545eea (127.29.62.193:36669): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ffd232a4798432b9cbceacdc65b20a9: Network error: Client connection negotiation failed: client connection to 127.29.62.193:36669: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:45:49.482352 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 4900
I20260808 17:45:49.488102 5350 ts_tablet_manager.cc:1939] T 1ffd232a4798432b9cbceacdc65b20a9 P 7ae3990f57fb42e3b3e8ca11c1ddedb4: Deleting consensus metadata
I20260808 17:45:49.488596 4577 catalog_manager.cc:4841] TS 7ae3990f57fb42e3b3e8ca11c1ddedb4 (127.29.62.195:33705): tablet 1ffd232a4798432b9cbceacdc65b20a9 (table test-table [id=eaf1085aa0704425b05019dc8acb4784]) successfully deleted
I20260808 17:45:49.501438 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 5092
I20260808 17:45:49.509074 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 4557
2026-08-08T17:45:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (784 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 17:45:49.535795 29947 mini_oidc.cc:152] Starting JWKS server
I20260808 17:45:49.536926 29947 webserver.cc:460] Webserver started at https://127.0.0.1:46149/ using document root <none> and password file <none>
I20260808 17:45:49.537005 29947 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 17:45:49.537137 29947 webserver.cc:460] Webserver started at http://127.0.0.1:34553/ using document root <none> and password file <none>
2026-08-08T17:45:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:49Z Disabled control of system clock
I20260808 17:45:49.546133 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:35295
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:36943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:35295
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46149/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/testchainca.cert with env {}
W20260808 17:45:49.627825 5358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:49.628031 5358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:49.628052 5358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:49.629148 5358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:49.629185 5358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:49.629197 5358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:49.629210 5358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:49.629222 5358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 17:45:49.629235 5358 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 17:45:49.629246 5358 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46149/jwks.json
I20260808 17:45:49.630820 5358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:35295
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:35295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46149/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:49.631028 5358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:49.631191 5358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:49.636092 5364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:49.636096 5366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:49.636098 5369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:49.636260 5358 server_base.cc:1034] running on GCE node
I20260808 17:45:49.636693 5358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:49.637038 5358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:49.638218 5358 hybrid_clock.cc:648] HybridClock initialized: now 1786211149638203 us; error 47 us; skew 500 ppm
I20260808 17:45:49.639464 5358 webserver.cc:460] Webserver started at http://127.29.62.254:37079/ using document root <none> and password file <none>
I20260808 17:45:49.639752 5358 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:49.639827 5358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:49.639989 5358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:49.641069 5358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "ea70677138624cfaa8bd918abd379b0e"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "608ce47dea76742ff9e2a60dcf588f8e"
server_key_iv: "ceddd9f508d7f877f5320f7899c56ee2"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.641456 5358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "ea70677138624cfaa8bd918abd379b0e"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "608ce47dea76742ff9e2a60dcf588f8e"
server_key_iv: "ceddd9f508d7f877f5320f7899c56ee2"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.642897 5358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:49.643635 5375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.643810 5358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:45:49.643929 5358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "ea70677138624cfaa8bd918abd379b0e"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "608ce47dea76742ff9e2a60dcf588f8e"
server_key_iv: "ceddd9f508d7f877f5320f7899c56ee2"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.644009 5358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:49.659837 5358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:49.660194 5358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:49.660362 5358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:49.664661 5358 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:35295
I20260808 17:45:49.664700 5468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:35295 every 8 connection(s)
I20260808 17:45:49.665071 5358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:49.666052 5469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:49.668530 5469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e: Bootstrap starting.
I20260808 17:45:49.669100 5469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:49.669341 5469 log.cc:824] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:49.669991 5469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e: No bootstrap required, opened a new log
I20260808 17:45:49.670966 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 5358
I20260808 17:45:49.671106 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:49.671329 29947 external_mini_cluster.cc:1428] Setting key 4aa6ce57c05c5e05d3c88c27e572a5a4
I20260808 17:45:49.672067 5469 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea70677138624cfaa8bd918abd379b0e" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35295 } }
I20260808 17:45:49.672257 5469 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:49.672292 5469 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea70677138624cfaa8bd918abd379b0e, State: Initialized, Role: FOLLOWER
I20260808 17:45:49.672421 5469 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea70677138624cfaa8bd918abd379b0e" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35295 } }
I20260808 17:45:49.672511 5469 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:49.672545 5469 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:49.672600 5469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:49.673606 5469 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea70677138624cfaa8bd918abd379b0e" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35295 } }
I20260808 17:45:49.673720 5469 leader_election.cc:302] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea70677138624cfaa8bd918abd379b0e; no voters:
I20260808 17:45:49.673866 5469 leader_election.cc:288] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:49.673961 5477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:49.674141 5477 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [term 1 LEADER]: Becoming Leader. State: Replica: ea70677138624cfaa8bd918abd379b0e, State: Running, Role: LEADER
I20260808 17:45:49.674160 5469 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:49.674280 5477 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea70677138624cfaa8bd918abd379b0e" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35295 } }
I20260808 17:45:49.675791 5478 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea70677138624cfaa8bd918abd379b0e. Latest consensus state: current_term: 1 leader_uuid: "ea70677138624cfaa8bd918abd379b0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea70677138624cfaa8bd918abd379b0e" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35295 } } }
I20260808 17:45:49.675877 5478 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:49.675999 5479 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea70677138624cfaa8bd918abd379b0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea70677138624cfaa8bd918abd379b0e" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35295 } } }
I20260808 17:45:49.676087 5479 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e [sys.catalog]: This master's current role is: LEADER
W20260808 17:45:49.676630 5503 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:45:49.676683 5503 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:45:49.676740 5500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:49.676939 5500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:49.678202 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.671883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.29.62.254:35295)
0808 17:45:49.672163 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.672172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.672205 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:45:49.672331 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.672338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.672406 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.672551 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.672899 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.674606 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.675820 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.676520 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.676643 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:49.677378 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.677381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.677382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.677550 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.677583 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.677593 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.677687 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.677765 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.677943 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":132,"threads_started":1}
I20260808 17:45:49.678583 5500 catalog_manager.cc:1347] Generated new cluster ID: e2bc6c7aebec4760a970465d16784c30
I20260808 17:45:49.678639 5500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:49.679158 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:45:49.685590 5500 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:49.686108 5500 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:49.692746 5500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea70677138624cfaa8bd918abd379b0e: Generated new TSK 0
I20260808 17:45:49.692943 5500 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 17:45:49.729669 5513 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 17:45:49.729930 5513 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 17:45:49.729961 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.706490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49486 (local address 127.29.62.254:35295)
0808 17:45:49.706563 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.706565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.706709 (+ 144us) server_negotiation.cc:407] Connection header received
0808 17:45:49.706820 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.706820 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.706831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.706892 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0808 17:45:49.707066 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.707549 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.708714 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.708900 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.708956 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:49.709196 (+ 240us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 17:45:49.729666 (+ 20470us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 17:45:49.729877 (+ 211us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49486: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 17:45:49.730033 5476 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49486: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 17:45:49.730448 5512 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:49.706415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:35295 (local address 127.0.0.1:49486)
0808 17:45:49.706630 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.706642 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.706719 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.706930 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.706931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.706933 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 17:45:49.706996 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:49.706999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.707652 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.707653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:49.708624 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:49.708627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.708995 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.708996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:49.709060 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:49.709085 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 17:45:49.729837 (+ 20752us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 17:45:49.729985 (+ 148us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.62.254:35295: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":122,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.62.254:35295: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 17:45:49.730866 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 5358
2026-08-08T17:45:49Z chronyd exiting
I20260808 17:45:49.774088 29947 test_util.cc:175] -----------------------------------------------
I20260808 17:45:49.774204 29947 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786211125631729-29947-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (239 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 17:45:49.775029 29947 mini_oidc.cc:152] Starting JWKS server
I20260808 17:45:49.775734 29947 webserver.cc:460] Webserver started at https://127.0.0.1:40587/ using document root <none> and password file <none>
I20260808 17:45:49.775807 29947 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 17:45:49.776000 29947 webserver.cc:460] Webserver started at http://127.0.0.1:39171/ using document root <none> and password file <none>
2026-08-08T17:45:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:49Z Disabled control of system clock
I20260808 17:45:49.784816 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:39047
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:45093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:39047
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40587/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/testchainca.cert with env {}
W20260808 17:45:49.865888 5525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:49.866089 5525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:49.866110 5525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:49.867295 5525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:49.867340 5525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:49.867353 5525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:49.867367 5525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:49.867432 5525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 17:45:49.867451 5525 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 17:45:49.867475 5525 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40587/jwks.json
I20260808 17:45:49.869000 5525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:39047
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:39047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40587/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:49.869257 5525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:49.869478 5525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:49.874361 5532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:49.874616 5525 server_base.cc:1034] running on GCE node
W20260808 17:45:49.874404 5537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:49.874369 5533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:49.874948 5525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:49.875232 5525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:49.876396 5525 hybrid_clock.cc:648] HybridClock initialized: now 1786211149876365 us; error 44 us; skew 500 ppm
I20260808 17:45:49.877693 5525 webserver.cc:460] Webserver started at http://127.29.62.254:39615/ using document root <none> and password file <none>
I20260808 17:45:49.877954 5525 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:49.878027 5525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:49.878177 5525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:49.879271 5525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "124307551758403ab8aaa76e86a5c557"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "f746e0dde7dbbd41807acd8cee1c94d6"
server_key_iv: "4706911d2278f982e6284441c5f2963f"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.879663 5525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "124307551758403ab8aaa76e86a5c557"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "f746e0dde7dbbd41807acd8cee1c94d6"
server_key_iv: "4706911d2278f982e6284441c5f2963f"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.880977 5525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:49.881706 5547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:49.881876 5525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:49.881987 5525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "124307551758403ab8aaa76e86a5c557"
format_stamp: "Formatted at 2026-08-08 17:45:49 on dist-test-slave-tk9z"
server_key: "f746e0dde7dbbd41807acd8cee1c94d6"
server_key_iv: "4706911d2278f982e6284441c5f2963f"
server_key_version: "encryptionkey@0"
I20260808 17:45:49.882062 5525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:49.892877 5525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:49.893151 5525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:49.893296 5525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:49.897425 5525 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:39047
I20260808 17:45:49.897450 5630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:39047 every 8 connection(s)
I20260808 17:45:49.897783 5525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:49.898505 5631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:49.899391 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 5525
I20260808 17:45:49.899510 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:49.899703 29947 external_mini_cluster.cc:1428] Setting key dd6ccaf7cdf1976baa50e7a6c436befc
I20260808 17:45:49.901883 5631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557: Bootstrap starting.
I20260808 17:45:49.902653 5631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:49.902961 5631 log.cc:824] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:49.903688 5631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557: No bootstrap required, opened a new log
I20260808 17:45:49.905215 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.900209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36616 (local address 127.29.62.254:39047)
0808 17:45:49.900406 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.900414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.900439 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:45:49.900841 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.900847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.900916 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.901028 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:49.901278 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.902185 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.903329 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.903805 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.903931 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:49.904569 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:49.904573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:49.904573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:49.904728 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:49.904760 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:49.904769 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:49.904820 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:49.904928 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 17:45:49.905011 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260808 17:45:49.906280 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:45:49.906157 5631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124307551758403ab8aaa76e86a5c557" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39047 } }
I20260808 17:45:49.906343 5631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:49.906378 5631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 124307551758403ab8aaa76e86a5c557, State: Initialized, Role: FOLLOWER
I20260808 17:45:49.906513 5631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124307551758403ab8aaa76e86a5c557" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39047 } }
I20260808 17:45:49.906587 5631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:49.906615 5631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:49.906661 5631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:49.907392 5631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124307551758403ab8aaa76e86a5c557" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39047 } }
I20260808 17:45:49.907531 5631 leader_election.cc:302] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 124307551758403ab8aaa76e86a5c557; no voters:
I20260808 17:45:49.907696 5631 leader_election.cc:288] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:49.907725 5641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:49.907871 5641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [term 1 LEADER]: Becoming Leader. State: Replica: 124307551758403ab8aaa76e86a5c557, State: Running, Role: LEADER
I20260808 17:45:49.907955 5631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:49.907958 5641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124307551758403ab8aaa76e86a5c557" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39047 } }
I20260808 17:45:49.908403 5642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "124307551758403ab8aaa76e86a5c557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124307551758403ab8aaa76e86a5c557" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39047 } } }
I20260808 17:45:49.908421 5643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 124307551758403ab8aaa76e86a5c557. Latest consensus state: current_term: 1 leader_uuid: "124307551758403ab8aaa76e86a5c557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "124307551758403ab8aaa76e86a5c557" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39047 } } }
I20260808 17:45:49.908515 5642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:49.908521 5643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:49.909049 5649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:49.909526 5649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:49.911043 5649 catalog_manager.cc:1347] Generated new cluster ID: 37f07649bc624983a52ef6893d54e7e8
I20260808 17:45:49.911098 5649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:49.930812 5649 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:49.931392 5649 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:49.941622 5649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 124307551758403ab8aaa76e86a5c557: Generated new TSK 0
I20260808 17:45:49.941839 5649 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 17:45:49.967192 5680 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 17:45:49.967288 5680 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 17:45:49.967617 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:49.953213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.29.62.254:39047)
0808 17:45:49.953281 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:49.953284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:49.953399 (+ 115us) server_negotiation.cc:407] Connection header received
0808 17:45:49.953492 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:49.953493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:49.953505 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:49.953567 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0808 17:45:49.953730 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.954259 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.955436 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.955633 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.955681 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:49.955874 (+ 193us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 17:45:49.967423 (+ 11549us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 17:45:49.967562 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36626: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 17:45:49.967700 5638 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36626: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 17:45:49.967809 5679 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:49.953155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:39047 (local address 127.0.0.1:36626)
0808 17:45:49.953324 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:49.953336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:49.953413 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:49.953592 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:49.953594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:49.953595 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 17:45:49.953665 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:49.953667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.954348 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.954349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:49.955341 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:49.955345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:49.955706 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:49.955707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:49.955772 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:49.955778 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 17:45:49.967548 (+ 11770us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 17:45:49.967695 (+ 147us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.62.254:39047: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":116,"threads_started":1}
I20260808 17:45:49.968140 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 5525
2026-08-08T17:45:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 17:45:50.013092 29947 mini_oidc.cc:152] Starting JWKS server
I20260808 17:45:50.013824 29947 webserver.cc:460] Webserver started at https://127.0.0.1:41319/ using document root <none> and password file <none>
I20260808 17:45:50.013868 29947 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 17:45:50.014065 29947 webserver.cc:460] Webserver started at http://127.0.0.1:37391/ using document root <none> and password file <none>
2026-08-08T17:45:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:50Z Disabled control of system clock
I20260808 17:45:50.022976 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:44401
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:44401
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41319/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 17:45:50.106112 5690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:50.106312 5690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:50.106334 5690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:50.107537 5690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:50.107579 5690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:50.107592 5690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:50.107605 5690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:50.107617 5690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 17:45:50.107630 5690 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 17:45:50.107641 5690 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41319/jwks.json
I20260808 17:45:50.108903 5690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:44401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:44401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41319/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:50.109086 5690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:50.109303 5690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:45:50.114138 5690 server_base.cc:1034] running on GCE node
W20260808 17:45:50.114085 5696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:50.114079 5697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:50.114078 5699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:50.114573 5690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:50.114867 5690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:50.116014 5690 hybrid_clock.cc:648] HybridClock initialized: now 1786211150116003 us; error 27 us; skew 500 ppm
I20260808 17:45:50.117240 5690 webserver.cc:460] Webserver started at http://127.29.62.254:45597/ using document root <none> and password file <none>
I20260808 17:45:50.117487 5690 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:50.117566 5690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:50.117722 5690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:50.118827 5690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "a95dc5baa9b64620ad65b74709790341"
format_stamp: "Formatted at 2026-08-08 17:45:50 on dist-test-slave-tk9z"
server_key: "7d9e1725722ce34e303e779e928b411e"
server_key_iv: "12ede2b7787abc74d86adfb4f383e97d"
server_key_version: "encryptionkey@0"
I20260808 17:45:50.119205 5690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "a95dc5baa9b64620ad65b74709790341"
format_stamp: "Formatted at 2026-08-08 17:45:50 on dist-test-slave-tk9z"
server_key: "7d9e1725722ce34e303e779e928b411e"
server_key_iv: "12ede2b7787abc74d86adfb4f383e97d"
server_key_version: "encryptionkey@0"
I20260808 17:45:50.120522 5690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:50.121212 5708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:50.121383 5690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:50.121500 5690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "a95dc5baa9b64620ad65b74709790341"
format_stamp: "Formatted at 2026-08-08 17:45:50 on dist-test-slave-tk9z"
server_key: "7d9e1725722ce34e303e779e928b411e"
server_key_iv: "12ede2b7787abc74d86adfb4f383e97d"
server_key_version: "encryptionkey@0"
I20260808 17:45:50.121580 5690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:50.132189 5690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:50.132498 5690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:50.132658 5690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:50.138011 5690 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:44401
I20260808 17:45:50.138048 5781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:44401 every 8 connection(s)
I20260808 17:45:50.138546 5690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:50.141355 5782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:50.145134 5782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341: Bootstrap starting.
I20260808 17:45:50.145869 5782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:50.146174 5782 log.cc:824] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:50.147065 5782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341: No bootstrap required, opened a new log
I20260808 17:45:50.147442 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 5690
I20260808 17:45:50.147576 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:50.147804 29947 external_mini_cluster.cc:1428] Setting key 57b43d0f5806c9641a145db4b8a16b34
I20260808 17:45:50.150084 5782 raft_consensus.cc:348] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95dc5baa9b64620ad65b74709790341" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44401 } }
I20260808 17:45:50.150295 5782 raft_consensus.cc:374] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:50.150343 5782 raft_consensus.cc:729] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a95dc5baa9b64620ad65b74709790341, State: Initialized, Role: FOLLOWER
I20260808 17:45:50.150511 5782 consensus_queue.cc:260] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95dc5baa9b64620ad65b74709790341" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44401 } }
I20260808 17:45:50.150601 5782 raft_consensus.cc:388] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:50.150640 5782 raft_consensus.cc:482] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:50.150682 5782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:50.151413 5782 raft_consensus.cc:504] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95dc5baa9b64620ad65b74709790341" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44401 } }
I20260808 17:45:50.151546 5782 leader_election.cc:302] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a95dc5baa9b64620ad65b74709790341; no voters:
I20260808 17:45:50.153180 5782 leader_election.cc:288] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:50.153200 5789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:50.154903 5789 raft_consensus.cc:686] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [term 1 LEADER]: Becoming Leader. State: Replica: a95dc5baa9b64620ad65b74709790341, State: Running, Role: LEADER
I20260808 17:45:50.155004 5789 consensus_queue.cc:237] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95dc5baa9b64620ad65b74709790341" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44401 } }
I20260808 17:45:50.155035 5782 sys_catalog.cc:564] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:50.155372 5790 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a95dc5baa9b64620ad65b74709790341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95dc5baa9b64620ad65b74709790341" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44401 } } }
I20260808 17:45:50.155467 5790 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:50.157107 5798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:50.157253 5791 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a95dc5baa9b64620ad65b74709790341. Latest consensus state: current_term: 1 leader_uuid: "a95dc5baa9b64620ad65b74709790341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95dc5baa9b64620ad65b74709790341" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 44401 } } }
I20260808 17:45:50.157316 5791 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:50.158221 5798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:50.158917 5788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:50.148356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51670 (local address 127.29.62.254:44401)
0808 17:45:50.148614 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:50.148622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:50.148657 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:45:50.150073 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:50.150083 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:50.150187 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:50.150390 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:50.150775 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.152377 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.153409 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.154277 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.154447 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:50.157904 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:50.157909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:50.157909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:50.158155 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:50.158208 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:50.158225 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:50.158297 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:50.158620 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 17:45:50.158704 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
I20260808 17:45:50.160123 5798 catalog_manager.cc:1347] Generated new cluster ID: 4a7d84e87c5247efb39effa36cd82e2a
I20260808 17:45:50.160156 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:45:50.160177 5798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:50.176842 5798 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:50.177667 5798 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:50.179970 5798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a95dc5baa9b64620ad65b74709790341: Generated new TSK 0
I20260808 17:45:50.180148 5798 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:45:50.194649 5788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:50.190582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51678 (local address 127.29.62.254:44401)
0808 17:45:50.190623 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:50.190626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:50.190798 (+ 172us) server_negotiation.cc:407] Connection header received
0808 17:45:50.190865 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:50.190866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:50.190876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:50.190969 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0808 17:45:50.191146 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.191646 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.192467 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.192695 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.192776 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:50.192959 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 17:45:50.194356 (+ 1397us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41319/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 17:45:50.194572 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41319/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 17:45:50.194715 5788 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41319/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 17:45:50.197793 5826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:50.190544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:44401 (local address 127.0.0.1:51678)
0808 17:45:50.190709 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:50.190721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:50.190795 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:50.191026 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:50.191028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:50.191030 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 17:45:50.191089 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:50.191091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.191762 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.191763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:50.192371 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:50.192373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.192783 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.192784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:50.192850 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:50.192855 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 17:45:50.194820 (+ 1965us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41319/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 17:45:50.197698 (+ 2878us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.62.254:44401: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41319/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":2670,"thread_start_us":98,"threads_started":1}
I20260808 17:45:50.198100 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 5690
E20260808 17:45:50.222568 5827 webserver.cc:533] Webserver: sslize failed:
E20260808 17:45:50.222659 5827 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T17:45:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T17:45:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:50Z Disabled control of system clock
I20260808 17:45:50.257681 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:46375
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:36115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:46375
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/testchainca.cert with env {}
W20260808 17:45:50.339530 5838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:50.339720 5838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:50.339741 5838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:50.340899 5838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:50.340947 5838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:50.340960 5838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:50.340972 5838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:50.340986 5838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 17:45:50.340997 5838 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 17:45:50.341040 5838 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 17:45:50.342530 5838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:46375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:46375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:50.342777 5838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:50.342998 5838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:50.347780 5856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:50.347896 5851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:50.347961 5838 server_base.cc:1034] running on GCE node
W20260808 17:45:50.347927 5848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:50.348301 5838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:50.348580 5838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:50.349725 5838 hybrid_clock.cc:648] HybridClock initialized: now 1786211150349705 us; error 38 us; skew 500 ppm
I20260808 17:45:50.350945 5838 webserver.cc:460] Webserver started at http://127.29.62.254:40401/ using document root <none> and password file <none>
I20260808 17:45:50.351151 5838 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:50.351190 5838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:50.351276 5838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:50.352286 5838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "d50a187d05734711b48864cfdc962241"
format_stamp: "Formatted at 2026-08-08 17:45:50 on dist-test-slave-tk9z"
server_key: "5febb7241004cd2e122fdf14e1fe5f9d"
server_key_iv: "082a869d5f1e2dfe7ee5be40fb339a8c"
server_key_version: "encryptionkey@0"
I20260808 17:45:50.352595 5838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "d50a187d05734711b48864cfdc962241"
format_stamp: "Formatted at 2026-08-08 17:45:50 on dist-test-slave-tk9z"
server_key: "5febb7241004cd2e122fdf14e1fe5f9d"
server_key_iv: "082a869d5f1e2dfe7ee5be40fb339a8c"
server_key_version: "encryptionkey@0"
I20260808 17:45:50.353811 5838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:50.354553 5864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:50.354735 5838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:50.354831 5838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "d50a187d05734711b48864cfdc962241"
format_stamp: "Formatted at 2026-08-08 17:45:50 on dist-test-slave-tk9z"
server_key: "5febb7241004cd2e122fdf14e1fe5f9d"
server_key_iv: "082a869d5f1e2dfe7ee5be40fb339a8c"
server_key_version: "encryptionkey@0"
I20260808 17:45:50.354928 5838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:50.368330 5838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:50.368621 5838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:50.368769 5838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:50.372952 5838 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:46375
I20260808 17:45:50.372987 5936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:46375 every 8 connection(s)
I20260808 17:45:50.373327 5838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:50.374326 5937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:50.376662 5937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241: Bootstrap starting.
I20260808 17:45:50.377228 5937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:50.377460 5937 log.cc:824] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:50.378118 5937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241: No bootstrap required, opened a new log
I20260808 17:45:50.379750 5937 raft_consensus.cc:348] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50a187d05734711b48864cfdc962241" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46375 } }
I20260808 17:45:50.379891 5937 raft_consensus.cc:374] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:50.379913 5937 raft_consensus.cc:729] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50a187d05734711b48864cfdc962241, State: Initialized, Role: FOLLOWER
I20260808 17:45:50.380023 5937 consensus_queue.cc:260] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50a187d05734711b48864cfdc962241" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46375 } }
I20260808 17:45:50.380077 5937 raft_consensus.cc:388] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:50.380097 5937 raft_consensus.cc:482] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:50.380123 5937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:50.380601 5937 raft_consensus.cc:504] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50a187d05734711b48864cfdc962241" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46375 } }
I20260808 17:45:50.380688 5937 leader_election.cc:302] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d50a187d05734711b48864cfdc962241; no voters:
I20260808 17:45:50.380811 5937 leader_election.cc:288] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:50.380873 5940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:50.381026 5940 raft_consensus.cc:686] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [term 1 LEADER]: Becoming Leader. State: Replica: d50a187d05734711b48864cfdc962241, State: Running, Role: LEADER
I20260808 17:45:50.381076 5937 sys_catalog.cc:564] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:50.381130 5940 consensus_queue.cc:237] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50a187d05734711b48864cfdc962241" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46375 } }
I20260808 17:45:50.381570 5943 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d50a187d05734711b48864cfdc962241. Latest consensus state: current_term: 1 leader_uuid: "d50a187d05734711b48864cfdc962241" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50a187d05734711b48864cfdc962241" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46375 } } }
I20260808 17:45:50.381672 5943 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:50.381840 5941 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d50a187d05734711b48864cfdc962241" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50a187d05734711b48864cfdc962241" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46375 } } }
I20260808 17:45:50.382098 5941 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:50.382171 5956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:50.382606 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 5838
I20260808 17:45:50.382710 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:50.382750 5956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:50.382910 29947 external_mini_cluster.cc:1428] Setting key 75c19d0e3a2ee7043805f53ecbd475b7
I20260808 17:45:50.384717 5956 catalog_manager.cc:1347] Generated new cluster ID: 70b19a954fc4479e8c0f044c9ad2110d
I20260808 17:45:50.384778 5956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:50.389117 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:50.383355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.29.62.254:46375)
0808 17:45:50.383716 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:50.383724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:50.383752 (+ 28us) server_negotiation.cc:407] Connection header received
0808 17:45:50.383867 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:50.383876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:50.383945 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:50.384081 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:50.384365 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.385693 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.386820 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.387479 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.387617 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:50.388349 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:50.388354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:50.388354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:45:50.388582 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:50.388629 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:45:50.388645 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:50.388721 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:50.388795 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 17:45:50.388879 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":90,"threads_started":1}
I20260808 17:45:50.390051 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:45:50.401844 5956 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:50.402570 5956 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:50.405664 5956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d50a187d05734711b48864cfdc962241: Generated new TSK 0
I20260808 17:45:50.405854 5956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:45:50.883450 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:50.412177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.29.62.254:46375)
0808 17:45:50.412245 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:50.412248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:50.412382 (+ 134us) server_negotiation.cc:407] Connection header received
0808 17:45:50.412442 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:50.412443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:50.412457 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:50.412537 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0808 17:45:50.412664 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.413485 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.414231 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.414590 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.414664 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:50.414823 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 17:45:50.883219 (+468396us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 17:45:50.883387 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45780: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 17:45:50.883522 5969 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45780: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 17:45:50.883649 5980 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:50.412102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:46375 (local address 127.0.0.1:45780)
0808 17:45:50.412299 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:50.412316 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:50.412373 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:50.412541 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:50.412542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:50.412543 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 17:45:50.412595 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:50.412597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.413556 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.413557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:50.414146 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:50.414147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:50.414671 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:50.414672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:50.414717 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:50.414721 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 17:45:50.883363 (+468642us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 17:45:50.883503 (+ 140us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.62.254:46375: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":113,"threads_started":1}
I20260808 17:45:50.884053 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 5838
2026-08-08T17:45:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (661 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (93 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:51 dist-test-slave-tk9z krb5kdc[6002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:51 dist-test-slave-tk9z krb5kdc[6002](info): set up 2 sockets
Aug 08 17:45:51 dist-test-slave-tk9z krb5kdc[6002](info): commencing operation
krb5kdc: starting...
W20260808 17:45:52.320320 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:52 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:45:52.444681 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:33959
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:34805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:33959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:52.527588 6023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:52.527819 6023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:52.527856 6023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:52.528995 6023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:52.529055 6023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:52.529080 6023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:52.529110 6023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:52.529142 6023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:52.530619 6023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:33959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:33959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:52.530890 6023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:52.531126 6023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:52.535866 6034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:52.535962 6023 server_base.cc:1034] running on GCE node
W20260808 17:45:52.535897 6032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:52.535882 6031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:52.536319 6023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:52.536609 6023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:52.537767 6023 hybrid_clock.cc:648] HybridClock initialized: now 1786211152537742 us; error 38 us; skew 500 ppm
Aug 08 17:45:52 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211152, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:52.540280 6023 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:45:52.540745 6023 webserver.cc:460] Webserver started at http://127.29.62.254:37927/ using document root <none> and password file <none>
I20260808 17:45:52.540998 6023 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:52.541064 6023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:52.541220 6023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:52.542243 6023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "d5c5b14da71347488c225f6ee29ab539"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "7e2aebf29920d4e0fb3615b4301002de"
server_key_iv: "5f0f03b01c38596b59bf00bd46d74cfb"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.542634 6023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "d5c5b14da71347488c225f6ee29ab539"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "7e2aebf29920d4e0fb3615b4301002de"
server_key_iv: "5f0f03b01c38596b59bf00bd46d74cfb"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.543977 6023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:52.544679 6042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:52.544862 6023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:52.544981 6023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "d5c5b14da71347488c225f6ee29ab539"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "7e2aebf29920d4e0fb3615b4301002de"
server_key_iv: "5f0f03b01c38596b59bf00bd46d74cfb"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.545056 6023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:52.556612 6023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:52.558836 6023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:52.559005 6023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:52.562995 6023 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:33959
I20260808 17:45:52.563047 6129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:33959 every 8 connection(s)
I20260808 17:45:52.563410 6023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:52.564460 6132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:52.566788 6132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539: Bootstrap starting.
I20260808 17:45:52.567317 6132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:52.567566 6132 log.cc:824] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:52.568207 6132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539: No bootstrap required, opened a new log
I20260808 17:45:52.569849 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 6023
I20260808 17:45:52.569767 6132 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c5b14da71347488c225f6ee29ab539" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33959 } }
I20260808 17:45:52.569912 6132 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:52.569978 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:52.569934 6132 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5c5b14da71347488c225f6ee29ab539, State: Initialized, Role: FOLLOWER
I20260808 17:45:52.570118 6132 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c5b14da71347488c225f6ee29ab539" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33959 } }
I20260808 17:45:52.570207 6132 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:52.570211 29947 external_mini_cluster.cc:1428] Setting key 5400c1d8b30afecad11c3f9e1a3a28f4
I20260808 17:45:52.570264 6132 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:52.570324 6132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:52.571457 6132 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c5b14da71347488c225f6ee29ab539" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33959 } }
I20260808 17:45:52.571570 6132 leader_election.cc:302] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5c5b14da71347488c225f6ee29ab539; no voters:
I20260808 17:45:52.571750 6132 leader_election.cc:288] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:52.571816 6137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:52.572006 6132 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:52.572034 6137 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [term 1 LEADER]: Becoming Leader. State: Replica: d5c5b14da71347488c225f6ee29ab539, State: Running, Role: LEADER
I20260808 17:45:52.572265 6137 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c5b14da71347488c225f6ee29ab539" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33959 } }
I20260808 17:45:52.572647 6138 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5c5b14da71347488c225f6ee29ab539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c5b14da71347488c225f6ee29ab539" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33959 } } }
I20260808 17:45:52.572724 6138 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:52.572670 6139 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5c5b14da71347488c225f6ee29ab539. Latest consensus state: current_term: 1 leader_uuid: "d5c5b14da71347488c225f6ee29ab539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5c5b14da71347488c225f6ee29ab539" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33959 } } }
I20260808 17:45:52.572775 6139 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:52.573048 6150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:52.573496 6150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:52.575392 6150 catalog_manager.cc:1347] Generated new cluster ID: 2a5a9a04a5ca41a8ad26ebf3d84d2b14
I20260808 17:45:52.575464 6150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:45:52 dist-test-slave-tk9z krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:52.582126 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:52.570738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.29.62.254:33959)
0808 17:45:52.571017 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:52.571029 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:52.571072 (+ 43us) server_negotiation.cc:407] Connection header received
0808 17:45:52.571207 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:52.571215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:52.571270 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:52.571418 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:52.572299 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.573723 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.574813 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.575416 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.575533 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:52.578416 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:52.578420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:52.578420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:52.578432 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:52.580473 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:52.580870 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:52.580873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:52.580873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:52.581006 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:52.581213 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:52.581214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:52.581214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:52.581527 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:52.581688 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:52.581759 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 17:45:52.581911 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":148,"threads_started":1}
I20260808 17:45:52.584218 6150 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:52.584877 6150 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:52.602944 6150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5c5b14da71347488c225f6ee29ab539: Generated new TSK 0
I20260808 17:45:52.603156 6150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:45:52.634996 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:33959
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:34805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:52.718421 6167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:52.718641 6167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:52.718688 6167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:52.719852 6167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:52.719914 6167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:52.719960 6167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:52.721441 6167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:33959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:52.721695 6167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:52.721923 6167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:52.727043 6177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:52.727048 6176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:52.727048 6179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:52.727360 6167 server_base.cc:1034] running on GCE node
I20260808 17:45:52.727556 6167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:52.727783 6167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:52.728929 6167 hybrid_clock.cc:648] HybridClock initialized: now 1786211152728921 us; error 32 us; skew 500 ppm
Aug 08 17:45:52 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211152, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:52.731590 6167 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:45:52.732034 6167 webserver.cc:460] Webserver started at http://127.29.62.193:36021/ using document root <none> and password file <none>
I20260808 17:45:52.732234 6167 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:52.732298 6167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:52.732450 6167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:52.733464 6167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "6b040998f45f4f61a6e527d48343eef3"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "3f4e23a62cd10bb9149078a3127b9f80"
server_key_iv: "7eea25039a7701645bd02ef2b37d3233"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.733776 6167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "6b040998f45f4f61a6e527d48343eef3"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "3f4e23a62cd10bb9149078a3127b9f80"
server_key_iv: "7eea25039a7701645bd02ef2b37d3233"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.735118 6167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:52.735786 6191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:52.735975 6167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:52.736038 6167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "6b040998f45f4f61a6e527d48343eef3"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "3f4e23a62cd10bb9149078a3127b9f80"
server_key_iv: "7eea25039a7701645bd02ef2b37d3233"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.736129 6167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:52.751314 6167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:52.753539 6167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:52.753655 6167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:52.753870 6167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:52.754186 6167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:52.754222 6167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:52.754243 6167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:52.754256 6167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:52.760290 6167 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:36601
I20260808 17:45:52.760532 6367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:36601 every 8 connection(s)
I20260808 17:45:52.760746 6167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:45:52 dist-test-slave-tk9z krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211152, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:52.769443 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:52.761483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:52379 (local address 127.29.62.254:33959)
0808 17:45:52.761556 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:52.761558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:52.761716 (+ 158us) server_negotiation.cc:407] Connection header received
0808 17:45:52.762653 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:52.762654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:52.762668 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:52.762719 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:52.763266 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.763766 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.764693 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.764890 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.764940 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:52.767043 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:52.767045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:52.767045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:52.767054 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:52.768283 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:52.768665 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:52.768666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:52.768666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:52.768720 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:52.768932 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:52.768933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:52.768933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:52.769035 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:52.769087 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:52.769310 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 17:45:52.769379 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:45:52.769603 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:52.761435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:33959 (local address 127.29.62.193:52379)
0808 17:45:52.761598 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:52.761632 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:52.762563 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:52.762773 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:52.762776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:52.763013 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:52.763193 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:52.763197 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.763848 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.763849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:52.764616 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:52.764618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.764961 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.764961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:52.765055 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:52.765556 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:52.765570 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:52.766960 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:52.768358 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:52.768365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:52.768367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:52.768595 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:52.768780 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:52.768780 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:52.768781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:52.768878 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:52.769090 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:52.769094 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:52.769202 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 17:45:52.769281 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:45:52.769395 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260808 17:45:52.770133 6368 heartbeater.cc:347] Connected to a master server at 127.29.62.254:33959
I20260808 17:45:52.770215 6368 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:52.770411 6368 heartbeater.cc:511] Master 127.29.62.254:33959 requested a full tablet report, sending...
I20260808 17:45:52.770721 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 6167
I20260808 17:45:52.770836 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:52.771031 29947 external_mini_cluster.cc:1428] Setting key 1564098c06fb21933eba52893851b5aa
I20260808 17:45:52.771014 6068 ts_manager.cc:194] Registered new tserver with Master: 6b040998f45f4f61a6e527d48343eef3 (127.29.62.193:36601)
I20260808 17:45:52.771890 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:52379
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:45:52.796962 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:33959
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:34805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:52.880110 6377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:52.880352 6377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:52.880393 6377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:52.881531 6377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:52.881599 6377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:52.881644 6377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:52.883212 6377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:33959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:52.883489 6377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:52.883728 6377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:52.888826 6385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:52.888821 6390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:52.888870 6377 server_base.cc:1034] running on GCE node
W20260808 17:45:52.888821 6386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:52.889200 6377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:52.889400 6377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:52.890566 6377 hybrid_clock.cc:648] HybridClock initialized: now 1786211152890557 us; error 28 us; skew 500 ppm
Aug 08 17:45:52 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211152, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:52.893088 6377 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:45:52.893530 6377 webserver.cc:460] Webserver started at http://127.29.62.194:44425/ using document root <none> and password file <none>
I20260808 17:45:52.893748 6377 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:52.893795 6377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:52.893939 6377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:52.895064 6377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "1cca98fd99014e94a1a708295d23794c"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "0aeda3bad73427dbb44c16b0dbec6c09"
server_key_iv: "e927aa3d4b9b1726646d809537f3ac55"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.895431 6377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "1cca98fd99014e94a1a708295d23794c"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "0aeda3bad73427dbb44c16b0dbec6c09"
server_key_iv: "e927aa3d4b9b1726646d809537f3ac55"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.896746 6377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:52.897481 6400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:52.897656 6377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:52.897761 6377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "1cca98fd99014e94a1a708295d23794c"
format_stamp: "Formatted at 2026-08-08 17:45:52 on dist-test-slave-tk9z"
server_key: "0aeda3bad73427dbb44c16b0dbec6c09"
server_key_iv: "e927aa3d4b9b1726646d809537f3ac55"
server_key_version: "encryptionkey@0"
I20260808 17:45:52.897835 6377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:52.929718 6377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:52.931716 6377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:52.931887 6377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:52.932143 6377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:52.932502 6377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:52.932557 6377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:52.932601 6377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:52.932632 6377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:52.938587 6377 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:37211
I20260808 17:45:52.938633 6602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:37211 every 8 connection(s)
I20260808 17:45:52.939002 6377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:52.942469 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 6377
I20260808 17:45:52.942592 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:52.942796 29947 external_mini_cluster.cc:1428] Setting key 20c78990fd1e0df19e663c9af1c64623
Aug 08 17:45:52 dist-test-slave-tk9z krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211152, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:52.949453 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:52.939823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:37421 (local address 127.29.62.254:33959)
0808 17:45:52.939901 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:52.939903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:52.940126 (+ 223us) server_negotiation.cc:407] Connection header received
0808 17:45:52.941085 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:52.941086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:52.941097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:52.941151 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:52.941782 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.942285 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.943716 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.943930 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.944008 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:52.946241 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:52.946242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:52.946243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:52.946251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:52.947851 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:52.948390 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:52.948391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:52.948391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:52.948455 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:52.948757 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:52.948758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:52.948759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:52.948887 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:52.948967 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:52.949250 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 17:45:52.949348 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:45:52.949653 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:52.939749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:33959 (local address 127.29.62.194:37421)
0808 17:45:52.939958 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:52.940013 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:52.940988 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:52.941208 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:52.941211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:52.941512 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:52.941702 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:52.941707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.942421 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.942422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:52.943636 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:52.943639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:52.944020 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:52.944021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:52.944169 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:52.944724 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:52.944732 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:52.946137 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:52.947985 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:52.947993 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:52.947995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:52.948293 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:52.948526 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:52.948527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:52.948528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:52.948688 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:52.948958 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:52.948965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:52.949115 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 17:45:52.949228 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 17:45:52.949402 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":111,"threads_started":1}
I20260808 17:45:52.950115 6603 heartbeater.cc:347] Connected to a master server at 127.29.62.254:33959
I20260808 17:45:52.950232 6603 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:52.950492 6603 heartbeater.cc:511] Master 127.29.62.254:33959 requested a full tablet report, sending...
I20260808 17:45:52.950893 6068 ts_manager.cc:194] Registered new tserver with Master: 1cca98fd99014e94a1a708295d23794c (127.29.62.194:37211)
I20260808 17:45:52.951287 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:37421
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:45:52.969069 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:33959
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:34805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:53.053334 6614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:53.053548 6614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:53.053572 6614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:53.054922 6614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:53.054972 6614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:53.055001 6614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:53.056524 6614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:33959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:53.056770 6614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:53.057008 6614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:53.062129 6625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:53.062121 6630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:53.062122 6622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:53.062212 6614 server_base.cc:1034] running on GCE node
I20260808 17:45:53.062573 6614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:53.062822 6614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:53.063963 6614 hybrid_clock.cc:648] HybridClock initialized: now 1786211153063953 us; error 28 us; skew 500 ppm
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:53.069269 6614 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:45:53.069738 6614 webserver.cc:460] Webserver started at http://127.29.62.195:39139/ using document root <none> and password file <none>
I20260808 17:45:53.069983 6614 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:53.070029 6614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:53.070194 6614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:53.071310 6614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "4b3ede8f73334d1e997f3e09be7ad4d9"
format_stamp: "Formatted at 2026-08-08 17:45:53 on dist-test-slave-tk9z"
server_key: "42a363c7491b91e9a310ade2deec2475"
server_key_iv: "9e4d8048ee90b46d5c670b819d1ce11a"
server_key_version: "encryptionkey@0"
I20260808 17:45:53.071702 6614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "4b3ede8f73334d1e997f3e09be7ad4d9"
format_stamp: "Formatted at 2026-08-08 17:45:53 on dist-test-slave-tk9z"
server_key: "42a363c7491b91e9a310ade2deec2475"
server_key_iv: "9e4d8048ee90b46d5c670b819d1ce11a"
server_key_version: "encryptionkey@0"
I20260808 17:45:53.073174 6614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:45:53.074018 6644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:53.074206 6614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:53.074285 6614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "4b3ede8f73334d1e997f3e09be7ad4d9"
format_stamp: "Formatted at 2026-08-08 17:45:53 on dist-test-slave-tk9z"
server_key: "42a363c7491b91e9a310ade2deec2475"
server_key_iv: "9e4d8048ee90b46d5c670b819d1ce11a"
server_key_version: "encryptionkey@0"
I20260808 17:45:53.074394 6614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:53.103650 6614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:53.105674 6614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:53.105791 6614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:53.106024 6614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:53.106436 6614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:53.106498 6614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:53.106541 6614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:53.106582 6614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:53.113256 6614 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:39963
I20260808 17:45:53.113302 6818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:39963 every 8 connection(s)
I20260808 17:45:53.113643 6614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:45:53.115016 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 6614
I20260808 17:45:53.115113 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:53.115299 29947 external_mini_cluster.cc:1428] Setting key 688949ed6331bbc3893a87c8f4c60e5f
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:45:53.124337 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:53.114430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:44717 (local address 127.29.62.254:33959)
0808 17:45:53.114485 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:53.114487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:53.114713 (+ 226us) server_negotiation.cc:407] Connection header received
0808 17:45:53.116114 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:53.116114 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:53.116127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:53.116181 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:53.116955 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.117475 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.119100 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.119299 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.119358 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:53.121408 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:53.121410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:53.121410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:53.121418 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:53.122879 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:53.123342 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:53.123344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:53.123344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:53.123407 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:53.123657 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:53.123658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:53.123659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:53.123776 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:53.123859 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:53.124141 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 17:45:53.124231 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:45:53.124509 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:53.114451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:33959 (local address 127.29.62.195:44717)
0808 17:45:53.114599 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:53.114637 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:53.115980 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:53.116257 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:53.116261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:53.116632 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:53.116839 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:53.116844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.117559 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.117559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:53.119020 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:53.119023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.119376 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.119377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:53.119478 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:53.119944 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:53.119952 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:53.121309 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:53.122980 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:53.122988 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:53.122990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:53.123214 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:53.123476 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:53.123477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:53.123477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:53.123585 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:53.123849 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:53.123856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:53.123998 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 17:45:53.124117 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 17:45:53.124271 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260808 17:45:53.125021 6820 heartbeater.cc:347] Connected to a master server at 127.29.62.254:33959
I20260808 17:45:53.125125 6820 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:53.125316 6820 heartbeater.cc:511] Master 127.29.62.254:33959 requested a full tablet report, sending...
I20260808 17:45:53.125713 6068 ts_manager.cc:194] Registered new tserver with Master: 4b3ede8f73334d1e997f3e09be7ad4d9 (127.29.62.195:39963)
I20260808 17:45:53.126078 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:44717
I20260808 17:45:53.127164 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/test-admin.keytab.
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:53.143402 29947 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 17:45:53.145223 6832 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:53.144415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56992 (local address 127.29.62.193:36601)
0808 17:45:53.144597 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:53.144602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:53.144611 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:53.144638 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:53.144640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:53.144653 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:53.144742 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:53.145136 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56992: BlockingRecv error: recv got EOF from 127.0.0.1:56992 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
W20260808 17:45:53.145329 29947 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5cc)
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:53.146315 29947 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:53.147301 29947 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:53.155515 6832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:53.147898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.29.62.193:36601)
0808 17:45:53.147959 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:53.147960 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:53.148022 (+ 62us) server_negotiation.cc:407] Connection header received
0808 17:45:53.148107 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:53.148108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:53.148124 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:53.148186 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:53.148658 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.149394 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.150156 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.150552 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.150635 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:53.152663 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:53.152673 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:53.152673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:53.152681 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:53.154171 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:53.154645 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:53.154648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:53.154649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:53.154711 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:53.154929 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:53.154930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:53.154930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:53.155077 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:53.155223 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:53.155309 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 17:45:53.155394 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 17:45:53.157299 6832 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:53.156704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.29.62.193:36601)
0808 17:45:53.156786 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:53.156788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:53.156824 (+ 36us) server_negotiation.cc:407] Connection header received
0808 17:45:53.156899 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:53.156899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:53.156911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:53.156960 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:53.157244 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57014: BlockingRecv error: recv got EOF from 127.0.0.1:57014 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 17:45:53.157363 29947 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5cc' of type FILE
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:53.158490 29947 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:53.159433 29947 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:53.165964 6832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:53.159940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57018 (local address 127.29.62.193:36601)
0808 17:45:53.159976 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:53.159978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:53.160006 (+ 28us) server_negotiation.cc:407] Connection header received
0808 17:45:53.160057 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:53.160057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:53.160064 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:53.160126 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:53.160456 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.160945 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.161583 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.161768 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.161817 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:53.163903 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:53.163904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:53.163904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:53.163910 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:53.165088 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:53.165392 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:53.165392 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:53.165393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:53.165443 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:53.165605 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:53.165606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:53.165606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:53.165693 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:53.165764 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:53.165846 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 17:45:53.165904 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:45:53.167699 6832 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:45:53.167048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.29.62.193:36601)
0808 17:45:53.167106 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:53.167108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:53.167115 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:45:53.167185 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:53.167186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:53.167193 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:53.167240 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:53.167641 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57024: BlockingRecv error: recv got EOF from 127.0.0.1:57024 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 17:45:53.167749 29947 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786211125631729-29947-0/krb5kdc/krb5cc' of type FILE
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:53.168735 29947 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:53.169561 29947 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:45:53.176414 6832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:53.170029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57034 (local address 127.29.62.193:36601)
0808 17:45:53.170080 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:53.170081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:53.170155 (+ 74us) server_negotiation.cc:407] Connection header received
0808 17:45:53.170190 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:53.170190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:53.170198 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:53.170242 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:53.170663 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.171111 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.171741 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:53.171985 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:53.172097 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:53.174165 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:53.174166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:53.174167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:53.174173 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:53.175432 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:53.175744 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:53.175744 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:53.175745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:53.175798 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:53.175962 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:53.175962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:53.175963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:53.176050 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:53.176119 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:53.176200 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 17:45:53.176256 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 17:45:53.177103 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 6167
I20260808 17:45:53.190188 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 6377
I20260808 17:45:53.206231 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 6614
I20260808 17:45:53.215058 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 6023
2026-08-08T17:45:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2143 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6854](info): set up 2 sockets
Aug 08 17:45:53 dist-test-slave-tk9z krb5kdc[6854](info): commencing operation
krb5kdc: starting...
W20260808 17:45:54.472931 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:54 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.254.keytab.
Entry for principal oryx/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.254.keytab.
I20260808 17:45:54.596982 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:42553
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.254.keytab
--principal=oryx/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:54.679329 6875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:54.679530 6875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:54.679553 6875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:54.680681 6875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:54.680722 6875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:54.680737 6875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:54.680775 6875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:54.680802 6875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:54.682320 6875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:42553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.254.keytab
--principal=oryx/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:42553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:54.682600 6875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:54.682826 6875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:54.687512 6885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:54.687656 6875 server_base.cc:1034] running on GCE node
W20260808 17:45:54.687502 6893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:54.687502 6890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:54.688030 6875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:54.688303 6875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:54.689463 6875 hybrid_clock.cc:648] HybridClock initialized: now 1786211154689455 us; error 40 us; skew 500 ppm
Aug 08 17:45:54 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:54.691958 6875 init.cc:377] Logged in from keytab as oryx/127.29.62.254@KRBTEST.COM (short username oryx)
I20260808 17:45:54.692406 6875 webserver.cc:460] Webserver started at http://127.29.62.254:34823/ using document root <none> and password file <none>
I20260808 17:45:54.692595 6875 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:54.692636 6875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:54.692723 6875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:54.693691 6875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "ef37bce3589b40349d7954d1bb59e228"
format_stamp: "Formatted at 2026-08-08 17:45:54 on dist-test-slave-tk9z"
server_key: "f3d19846788fb87919896289a42eb25f"
server_key_iv: "85cc7b64a3f0bbbf2a90cd7cecdd5524"
server_key_version: "encryptionkey@0"
I20260808 17:45:54.693998 6875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "ef37bce3589b40349d7954d1bb59e228"
format_stamp: "Formatted at 2026-08-08 17:45:54 on dist-test-slave-tk9z"
server_key: "f3d19846788fb87919896289a42eb25f"
server_key_iv: "85cc7b64a3f0bbbf2a90cd7cecdd5524"
server_key_version: "encryptionkey@0"
I20260808 17:45:54.695369 6875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:54.696096 6902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:54.696272 6875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:45:54.696388 6875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "ef37bce3589b40349d7954d1bb59e228"
format_stamp: "Formatted at 2026-08-08 17:45:54 on dist-test-slave-tk9z"
server_key: "f3d19846788fb87919896289a42eb25f"
server_key_iv: "85cc7b64a3f0bbbf2a90cd7cecdd5524"
server_key_version: "encryptionkey@0"
I20260808 17:45:54.696467 6875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:54.704716 6875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:54.706461 6875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:54.706612 6875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:54.710516 6875 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:42553
I20260808 17:45:54.710564 6996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:42553 every 8 connection(s)
I20260808 17:45:54.710927 6875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:54.711880 6998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:54.712245 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 6875
I20260808 17:45:54.712378 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:54.712597 29947 external_mini_cluster.cc:1428] Setting key d9fbb26c52a5925333a348a38e049875
I20260808 17:45:54.715155 6998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: Bootstrap starting.
I20260808 17:45:54.715758 6998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:54.716095 6998 log.cc:824] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:54.716859 6998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: No bootstrap required, opened a new log
Aug 08 17:45:54 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:45:54.719439 6998 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } }
I20260808 17:45:54.719645 6998 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:54.719694 6998 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef37bce3589b40349d7954d1bb59e228, State: Initialized, Role: FOLLOWER
I20260808 17:45:54.719828 6998 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } }
I20260808 17:45:54.719941 6998 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:54.720006 6998 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:54.720064 6998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:54.720798 6998 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } }
I20260808 17:45:54.720938 6998 leader_election.cc:302] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef37bce3589b40349d7954d1bb59e228; no voters:
I20260808 17:45:54.721120 6998 leader_election.cc:288] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:54.721177 7006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:54.721344 7006 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 1 LEADER]: Becoming Leader. State: Replica: ef37bce3589b40349d7954d1bb59e228, State: Running, Role: LEADER
I20260808 17:45:54.721397 6998 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:54.721479 7006 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } }
I20260808 17:45:54.721797 7006 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef37bce3589b40349d7954d1bb59e228. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } }
I20260808 17:45:54.721895 7006 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:54.721808 7012 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } }
I20260808 17:45:54.722055 7012 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:54.722172 7020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:54.722772 7020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:54.723948 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:54.713104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.29.62.254:42553)
0808 17:45:54.713313 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:54.713322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:54.713382 (+ 60us) server_negotiation.cc:407] Connection header received
0808 17:45:54.713839 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:54.713845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:54.713887 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:54.714013 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:54.714838 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:54.715752 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:54.716755 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:54.717187 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:54.717276 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:54.719919 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:54.719923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:54.719924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:54.719936 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:54.722079 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:54.722560 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:54.722563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:54.722563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:54.722661 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:54.722899 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:54.722900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:54.722900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:54.723199 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:54.723326 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:54.723590 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 17:45:54.723761 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260808 17:45:54.727330 7020 catalog_manager.cc:1347] Generated new cluster ID: 11c55d0c6a7040a39622e80b0dfcf358
I20260808 17:45:54.727375 7020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:54.741686 7020 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:54.742220 7020 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:54.762231 7020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: Generated new TSK 0
I20260808 17:45:54.762470 7020 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab.
Entry for principal oryx/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab.
I20260808 17:45:54.800659 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42553
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab
--principal=oryx/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:54.901177 7042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:54.901377 7042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:54.901400 7042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:54.902629 7042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:54.902671 7042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:54.902719 7042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:54.904201 7042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab
--principal=oryx/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:54.904451 7042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:54.904687 7042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:54.910604 7051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:54.910670 7052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:54.910904 7055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:54.911128 7042 server_base.cc:1034] running on GCE node
I20260808 17:45:54.911387 7042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:54.911660 7042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:54.912806 7042 hybrid_clock.cc:648] HybridClock initialized: now 1786211154912786 us; error 39 us; skew 500 ppm
Aug 08 17:45:54 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:54.915606 7042 init.cc:377] Logged in from keytab as oryx/127.29.62.193@KRBTEST.COM (short username oryx)
I20260808 17:45:54.916090 7042 webserver.cc:460] Webserver started at http://127.29.62.193:34655/ using document root <none> and password file <none>
I20260808 17:45:54.916320 7042 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:54.916390 7042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:54.916545 7042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:54.917603 7042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "b9c78ccb5ccf415c96be50fd36c64f17"
format_stamp: "Formatted at 2026-08-08 17:45:54 on dist-test-slave-tk9z"
server_key: "65be4391220e81490333f31c2bf3a225"
server_key_iv: "dd9729a19962f39655fb4235f94d2cfe"
server_key_version: "encryptionkey@0"
I20260808 17:45:54.917991 7042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "b9c78ccb5ccf415c96be50fd36c64f17"
format_stamp: "Formatted at 2026-08-08 17:45:54 on dist-test-slave-tk9z"
server_key: "65be4391220e81490333f31c2bf3a225"
server_key_iv: "dd9729a19962f39655fb4235f94d2cfe"
server_key_version: "encryptionkey@0"
I20260808 17:45:54.919414 7042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:54.920141 7063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:54.920312 7042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:54.920420 7042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "b9c78ccb5ccf415c96be50fd36c64f17"
format_stamp: "Formatted at 2026-08-08 17:45:54 on dist-test-slave-tk9z"
server_key: "65be4391220e81490333f31c2bf3a225"
server_key_iv: "dd9729a19962f39655fb4235f94d2cfe"
server_key_version: "encryptionkey@0"
I20260808 17:45:54.920496 7042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:54.939694 7042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:54.941656 7042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:54.941838 7042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:54.942085 7042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:54.942471 7042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:54.942529 7042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:54.942572 7042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:54.942600 7042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:54.948779 7042 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:42193
I20260808 17:45:54.948818 7249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:42193 every 8 connection(s)
I20260808 17:45:54.949172 7042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:45:54 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:45:54.957118 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 7042
I20260808 17:45:54.957245 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:45:54.957463 29947 external_mini_cluster.cc:1428] Setting key 4f9469bb0824ab632919d93601d9880f
I20260808 17:45:54.958688 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:54.949939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:47869 (local address 127.29.62.254:42553)
0808 17:45:54.950041 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:54.950043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:54.950272 (+ 229us) server_negotiation.cc:407] Connection header received
0808 17:45:54.950805 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:54.950806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:54.950817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:54.950870 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:54.951835 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:54.952333 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:54.953285 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:54.953486 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:54.953537 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:54.955679 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:54.955681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:54.955681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:54.955688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:54.956942 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:54.957476 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:54.957477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:54.957478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:54.957554 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:54.957915 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:54.957915 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:54.957916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:54.958045 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:54.958133 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:54.958503 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 17:45:54.958594 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 17:45:54.958896 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:54.949905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.193:47869)
0808 17:45:54.950132 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:54.950172 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:54.950713 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:54.950936 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:54.950939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:54.951548 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:54.951749 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:54.951754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:54.952415 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:54.952416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:54.953206 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:54.953209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:54.953561 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:54.953562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:54.953665 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:54.954129 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:54.954137 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:54.955569 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:54.957043 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:54.957049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:54.957051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:54.957392 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:54.957680 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:54.957681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:54.957681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:54.957846 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:54.958190 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:54.958197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:54.958324 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 17:45:54.958468 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 17:45:54.958647 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":121,"threads_started":1}
I20260808 17:45:54.959384 7250 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42553
I20260808 17:45:54.959484 7250 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:54.959686 7250 heartbeater.cc:511] Master 127.29.62.254:42553 requested a full tablet report, sending...
I20260808 17:45:54.960290 6927 ts_manager.cc:194] Registered new tserver with Master: b9c78ccb5ccf415c96be50fd36c64f17 (127.29.62.193:42193)
I20260808 17:45:54.960955 6927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:47869
WARNING: no policy specified for oryx/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab.
Entry for principal oryx/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab.
I20260808 17:45:54.983062 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42553
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab
--principal=oryx/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:55.066138 7257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:55.066365 7257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:55.066429 7257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:55.067615 7257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:55.067693 7257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:55.067739 7257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:55.069172 7257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab
--principal=oryx/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:55.069443 7257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:55.069679 7257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:55.074689 7266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.074730 7267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.074774 7257 server_base.cc:1034] running on GCE node
W20260808 17:45:55.074707 7271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.075143 7257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:55.075410 7257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:55.076555 7257 hybrid_clock.cc:648] HybridClock initialized: now 1786211155076545 us; error 31 us; skew 500 ppm
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:55.079053 7257 init.cc:377] Logged in from keytab as oryx/127.29.62.194@KRBTEST.COM (short username oryx)
I20260808 17:45:55.079461 7257 webserver.cc:460] Webserver started at http://127.29.62.194:32851/ using document root <none> and password file <none>
I20260808 17:45:55.079689 7257 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.079753 7257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.079905 7257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:55.080888 7257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "cb4adc4acd0546f79732e4e9430c2484"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "c8b09523c1a9f9a2261729588f1d3ed0"
server_key_iv: "f5ec005443990bd242f92d0d86cea002"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.081252 7257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "cb4adc4acd0546f79732e4e9430c2484"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "c8b09523c1a9f9a2261729588f1d3ed0"
server_key_iv: "f5ec005443990bd242f92d0d86cea002"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.082638 7257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:45:55.083381 7281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.083552 7257 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:55.083657 7257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "cb4adc4acd0546f79732e4e9430c2484"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "c8b09523c1a9f9a2261729588f1d3ed0"
server_key_iv: "f5ec005443990bd242f92d0d86cea002"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.083730 7257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.105615 7257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:55.108125 7257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:55.108292 7257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:55.108577 7257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:55.108906 7257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:55.108959 7257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.109005 7257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:55.109036 7257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.115396 7257 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:44579
I20260808 17:45:55.115442 7466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:44579 every 8 connection(s)
I20260808 17:45:55.115774 7257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:55.118685 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 7257
I20260808 17:45:55.118800 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:45:55.119005 29947 external_mini_cluster.cc:1428] Setting key e29abf09eb83d3880c3d0372a53714fa
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:45:55.126549 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.116520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:59157 (local address 127.29.62.254:42553)
0808 17:45:55.116589 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.116592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.116814 (+ 222us) server_negotiation.cc:407] Connection header received
0808 17:45:55.117305 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.117306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.117316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.117378 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.118301 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.119112 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.120373 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.120696 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.120801 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.123249 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.123251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.123251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.123259 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.125060 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.125466 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.125467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.125468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.125539 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.125780 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.125781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.125781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.125935 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.126016 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.126258 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.126359 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:45:55.126567 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.116512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.194:59157)
0808 17:45:55.116678 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.116721 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.117224 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.117439 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.117442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.118016 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.118217 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.118221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.119239 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.119241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.120266 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.120269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.120878 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.120880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.121031 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.121599 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.121611 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.123156 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.125153 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.125159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.125161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.125377 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.125613 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.125613 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.125614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.125712 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.126009 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.126013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.126124 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.126205 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.126321 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260808 17:45:55.127079 7467 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42553
I20260808 17:45:55.127197 7467 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:55.127391 7467 heartbeater.cc:511] Master 127.29.62.254:42553 requested a full tablet report, sending...
I20260808 17:45:55.127835 6927 ts_manager.cc:194] Registered new tserver with Master: cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579)
I20260808 17:45:55.128209 6927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:59157
WARNING: no policy specified for oryx/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab.
Entry for principal oryx/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab.
I20260808 17:45:55.144917 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42553
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab
--principal=oryx/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:55.227861 7475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:55.228098 7475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:55.228140 7475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:55.229293 7475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:55.229358 7475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:55.229403 7475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:55.230935 7475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab
--principal=oryx/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:55.231184 7475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:55.231407 7475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:55.236238 7482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.236367 7475 server_base.cc:1034] running on GCE node
W20260808 17:45:55.236244 7485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.236459 7481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.236660 7475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:55.236894 7475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:55.238075 7475 hybrid_clock.cc:648] HybridClock initialized: now 1786211155238059 us; error 33 us; skew 500 ppm
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:55.240720 7475 init.cc:377] Logged in from keytab as oryx/127.29.62.195@KRBTEST.COM (short username oryx)
I20260808 17:45:55.241096 7475 webserver.cc:460] Webserver started at http://127.29.62.195:37859/ using document root <none> and password file <none>
I20260808 17:45:55.241268 7475 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.241308 7475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.241396 7475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:55.242422 7475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "be99fa38df004be29e01f1dd07c81b41"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b50f2df6eb75fef32f179ae73d1ebc1d"
server_key_iv: "3db3c1d3e0534b7ec3d76979ec8f92c8"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.242764 7475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "be99fa38df004be29e01f1dd07c81b41"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b50f2df6eb75fef32f179ae73d1ebc1d"
server_key_iv: "3db3c1d3e0534b7ec3d76979ec8f92c8"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.244037 7475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:55.244707 7495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.244877 7475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:55.244995 7475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "be99fa38df004be29e01f1dd07c81b41"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b50f2df6eb75fef32f179ae73d1ebc1d"
server_key_iv: "3db3c1d3e0534b7ec3d76979ec8f92c8"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.245072 7475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.267925 7475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:55.270530 7475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:55.270701 7475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:55.270960 7475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:55.271302 7475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:55.271358 7475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.271404 7475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:55.271428 7475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.277248 7475 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:45125
I20260808 17:45:55.277330 7677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:45125 every 8 connection(s)
I20260808 17:45:55.277664 7475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:45:55.280134 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 7475
I20260808 17:45:55.280251 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:45:55.280437 29947 external_mini_cluster.cc:1428] Setting key 9f2507dcc15fd4d9053db0cd17349637
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:45:55.287127 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.278452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:45703 (local address 127.29.62.254:42553)
0808 17:45:55.278517 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.278519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.278718 (+ 199us) server_negotiation.cc:407] Connection header received
0808 17:45:55.279274 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.279276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.279289 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.279353 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.280347 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.281070 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.282087 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.282278 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.282335 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.284381 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.284383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.284383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.284390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.285866 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.286267 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.286268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.286269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.286349 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.286624 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.286624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.286625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.286732 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.286784 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.287005 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.287065 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:45:55.287405 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.278442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.195:45703)
0808 17:45:55.278599 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.278635 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.279165 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.279424 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.279428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.280032 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.280225 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.280231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.281156 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.281157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.282006 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.282009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.282354 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.282355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.282524 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.282959 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.282967 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.284296 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.285947 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.285953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.285955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.286165 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.286464 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.286465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.286465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.286565 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.286789 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.286793 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.286910 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.286992 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.287153 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260808 17:45:55.287870 7679 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42553
I20260808 17:45:55.287972 7679 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:55.288146 7679 heartbeater.cc:511] Master 127.29.62.254:42553 requested a full tablet report, sending...
I20260808 17:45:55.288511 6927 ts_manager.cc:194] Registered new tserver with Master: be99fa38df004be29e01f1dd07c81b41 (127.29.62.195:45125)
I20260808 17:45:55.288872 6927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:45703
I20260808 17:45:55.292308 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.62.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.253.keytab.
Entry for principal oryx/127.29.62.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.253.keytab.
I20260808 17:45:55.316759 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.253:44951
--webserver_interface=127.29.62.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.253.keytab
--principal=oryx/127.29.62.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:42553,127.29.62.253:44951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:55.398012 7688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:55.398224 7688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:55.398247 7688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:55.399430 7688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:55.399474 7688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:55.399489 7688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:55.399526 7688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:55.399559 7688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:55.401058 7688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:42553,127.29.62.253:44951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.253.keytab
--principal=oryx/127.29.62.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.253:44951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.62.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
W20260808 17:45:55.401201 7688 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.62.254:42553,127.29.62.253:44951'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 17:45:55.401360 7688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:55.401600 7688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:55.406287 7700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.406267 7696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.406275 7695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.406637 7688 server_base.cc:1034] running on GCE node
I20260808 17:45:55.406910 7688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:55.407256 7688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:55.408419 7688 hybrid_clock.cc:648] HybridClock initialized: now 1786211155408410 us; error 47 us; skew 500 ppm
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:55.411024 7688 init.cc:377] Logged in from keytab as oryx/127.29.62.253@KRBTEST.COM (short username oryx)
I20260808 17:45:55.411422 7688 webserver.cc:460] Webserver started at http://127.29.62.253:43565/ using document root <none> and password file <none>
I20260808 17:45:55.411655 7688 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.411727 7688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.411887 7688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:55.412942 7688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data/instance:
uuid: "bac033c9a915486d8910ac4bbc5636dd"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "38b9724c5d9a1d5e2ccd0c5992b6787a"
server_key_iv: "6a7fdf2ed45b7c27a1bf5987297d02a0"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.413327 7688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal/instance:
uuid: "bac033c9a915486d8910ac4bbc5636dd"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "38b9724c5d9a1d5e2ccd0c5992b6787a"
server_key_iv: "6a7fdf2ed45b7c27a1bf5987297d02a0"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.414729 7688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:55.415467 7709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.415644 7688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:45:55.415747 7688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
uuid: "bac033c9a915486d8910ac4bbc5636dd"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "38b9724c5d9a1d5e2ccd0c5992b6787a"
server_key_iv: "6a7fdf2ed45b7c27a1bf5987297d02a0"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.415830 7688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.432493 7688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:55.434825 7688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:55.434995 7688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:55.438894 7688 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.253:44951
I20260808 17:45:55.438930 7784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.253:44951 every 8 connection(s)
I20260808 17:45:55.439321 7688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
I20260808 17:45:55.440320 7785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:55.442016 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 7688
I20260808 17:45:55.442001 7785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.253@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.62.253@KRBTEST.COM
I20260808 17:45:55.452814 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.442883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45500 (local address 127.29.62.254:42553)
0808 17:45:55.442959 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.442961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.443133 (+ 172us) server_negotiation.cc:407] Connection header received
0808 17:45:55.443632 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.443634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.443646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.443705 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.444449 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.445242 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.446496 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.446797 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.446871 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.449210 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.449212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.449212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.449220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.451072 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.451756 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.451757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.451758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.451845 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.452109 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.452110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.452110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.452276 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.452351 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.452619 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.452710 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 17:45:55.453114 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.442824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:45500)
0808 17:45:55.443029 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.443068 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.443541 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.443731 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.443733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.444108 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.444353 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.444359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.445342 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.445343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.446404 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.446408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.446882 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.446882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.446991 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.447471 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.447499 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.449116 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.451169 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.451176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.451178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.451641 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.451922 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.451923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.451924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.452038 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.452353 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.452360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.452487 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.452581 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.452733 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":68,"threads_started":1}
I20260808 17:45:55.453115 7788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.442755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51760 (local address 127.29.62.253:44951)
0808 17:45:55.442943 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.442952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.443070 (+ 118us) server_negotiation.cc:407] Connection header received
0808 17:45:55.443557 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.443566 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.443606 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.443710 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.444454 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.445559 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.446568 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.447011 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.447061 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.449817 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.449825 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.449826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.449839 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.451668 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.452039 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.452042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.452042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.452103 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.452311 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.452312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.452312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.452491 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.452646 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.452732 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.452814 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
I20260808 17:45:55.453704 7785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.253@KRBTEST.COM for oryx/127.29.62.253@KRBTEST.COM
I20260808 17:45:55.459702 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.453986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.0.0.1:51762)
0808 17:45:55.454019 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.454027 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.454060 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.454166 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.454166 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.454270 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.454305 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.454306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.454923 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.454924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.455403 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.455405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.455753 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.455754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.455798 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.456203 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.456208 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.457306 (+ 1098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.458694 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.458695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.458695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.458866 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.459146 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.459146 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.459147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.459206 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.459485 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.459486 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.459517 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.459568 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.459644 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 17:45:55.459817 7788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.454019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51762 (local address 127.29.62.253:44951)
0808 17:45:55.454069 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.454071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.454079 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:45:55.454097 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.454098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.454107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.454161 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.454358 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.454849 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.455471 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.455677 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.455743 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.457379 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.457380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.457380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.457385 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.458624 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.458927 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.458928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.458928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.458979 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.459281 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.459282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.459282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.459422 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.459483 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.459602 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.459690 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":32}
I20260808 17:45:55.460979 7785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Bootstrap starting.
I20260808 17:45:55.461807 7785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:55.462111 7785 log.cc:824] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:55.462883 7785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: No bootstrap required, opened a new log
I20260808 17:45:55.464254 7785 raft_consensus.cc:348] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } }
I20260808 17:45:55.464375 7785 raft_consensus.cc:374] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:55.464423 7785 raft_consensus.cc:729] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bac033c9a915486d8910ac4bbc5636dd, State: Initialized, Role: FOLLOWER
I20260808 17:45:55.464548 7785 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } }
I20260808 17:45:55.464847 7785 sys_catalog.cc:564] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:55.464804 7794 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } } }
I20260808 17:45:55.464947 7794 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:45:55.466503 7813 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:45:55.466558 7813 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:45:55.468222 7688 master_runner.cc:428] Detected that this master bac033c9a915486d8910ac4bbc5636dd is joining an existing cluster
I20260808 17:45:55.468307 7688 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.62.253:44951
I20260808 17:45:55.476009 7788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.469302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.29.62.253:44951)
0808 17:45:55.469344 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.469347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.469549 (+ 202us) server_negotiation.cc:407] Connection header received
0808 17:45:55.469616 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.469618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.469624 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.469669 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.470087 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.470854 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.471925 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.472109 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.472190 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.473147 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.473149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.473150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.473157 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.474830 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.475212 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.475213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.475213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.475262 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.475481 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.475481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.475482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.475601 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.475645 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.475834 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.475905 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:45:55.476298 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.469254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.0.0.1:51764)
0808 17:45:55.469479 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.469490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.469570 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.469676 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.469676 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.469933 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.470000 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.470002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.470941 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.470942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.471838 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.471841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.472200 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.472201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.472261 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.472685 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.472690 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.473060 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.474923 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.474924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.474924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.475142 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.475351 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.475351 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.475352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.475413 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.475685 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.475686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.475719 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.476084 (+ 365us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.476182 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":2,"thread_start_us":175,"threads_started":1}
I20260808 17:45:55.477346 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.469312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45508 (local address 127.29.62.254:42553)
0808 17:45:55.469348 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.469350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.469488 (+ 138us) server_negotiation.cc:407] Connection header received
0808 17:45:55.469558 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.469559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.469568 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.469621 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.470053 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.470860 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.472873 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.473179 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.473260 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.474323 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.474324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.474325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.474331 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.476077 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.476552 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.476553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.476554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.476636 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.476832 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.476833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.476833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.476961 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.477026 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.477184 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.477257 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:45:55.477372 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.469253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:45508)
0808 17:45:55.469435 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.469445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.469502 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.469700 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.469700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.469915 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.469978 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.469979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.471944 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.471944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.472781 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.472784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.473273 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.473274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.473341 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.473771 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.473776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.474223 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.476195 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.476196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.476196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.476397 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.476713 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.476713 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.476714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.476778 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.477016 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.477017 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.477069 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.477158 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.477258 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.254@KRBTEST.COM for oryx/127.29.62.253@KRBTEST.COM
I20260808 17:45:55.486244 7788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.479465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.29.62.253:44951)
0808 17:45:55.479504 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.479505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.479618 (+ 113us) server_negotiation.cc:407] Connection header received
0808 17:45:55.479703 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.479704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.479714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.479767 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.480206 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.480628 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.481358 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.481551 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.481624 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.483595 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.483596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.483597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.483604 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.485034 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.485422 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.485423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.485423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.485481 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.485707 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.485708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.485708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.485832 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.485893 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.486080 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.486157 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:45:55.486261 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.479386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.0.0.1:51772)
0808 17:45:55.479510 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.479551 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.479641 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.479773 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.479776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.480077 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.480129 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.480130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.480707 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.480708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.481274 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.481276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.481624 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.481625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.481700 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.482152 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.482156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.483504 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.485130 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.485138 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.485140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.485335 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.485554 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.485555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.485556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.485617 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.485899 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.485908 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.485958 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.486054 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.486167 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260808 17:45:55.486871 6927 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.62.253:44951: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd"
member_type: NON_VOTER
last_known_addr {
host: "127.29.62.253"
port: 44951
}
attrs {
promote: true
}
}
dest_uuid: "ef37bce3589b40349d7954d1bb59e228"
cas_config_opid_index: -1
I20260808 17:45:55.487226 6927 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } }
I20260808 17:45:55.487709 7835 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bac033c9a915486d8910ac4bbc5636dd (127.29.62.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } }
I20260808 17:45:55.488044 7748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:55.488721 7835 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.62.253:44951
I20260808 17:45:55.488881 7748 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Refusing update from remote peer ef37bce3589b40349d7954d1bb59e228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 17:45:55.488901 7833 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } }
I20260808 17:45:55.488992 7833 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:55.488991 7688 master.cc:537] Master@127.29.62.253:44951 shutting down...
I20260808 17:45:55.489214 7833 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 17:45:55.489936 6906 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 17:45:55.490527 6906 consensus_peers.cc:487] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 -> Peer bac033c9a915486d8910ac4bbc5636dd (127.29.62.253:44951): Couldn't send request to peer bac033c9a915486d8910ac4bbc5636dd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 17:45:55.492170 7688 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:55.492261 7688 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:55.492311 7688 tablet_replica.cc:330] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: stopping tablet replica
I20260808 17:45:55.505121 7688 master.cc:552] Master@127.29.62.253:44951 shutdown complete.
I20260808 17:45:55.505178 7688 master_runner.cc:305] Clearing existing system tablet
I20260808 17:45:55.505314 7688 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.505365 7688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.506112 7688 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:45:55.506721 7841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.506881 7688 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.506946 7688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.506979 7688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
uuid: "bac033c9a915486d8910ac4bbc5636dd"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "38b9724c5d9a1d5e2ccd0c5992b6787a"
server_key_iv: "6a7fdf2ed45b7c27a1bf5987297d02a0"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.507848 7688 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:55.509686 7688 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 17:45:55.509761 7688 log.cc:1196] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 17:45:55.509950 7688 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Deleting consensus metadata
I20260808 17:45:55.510131 7688 master_runner.cc:315] Copying system tablet from 127.29.62.254:42553
I20260808 17:45:55.510792 7688 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: tablet copy: Beginning tablet copy session from remote peer at address 127.29.62.254:42553
I20260808 17:45:55.517426 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.511094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:45514)
0808 17:45:55.511209 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.511220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.511288 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.511433 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.511434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.511601 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.511649 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.511651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.512613 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.512614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.513183 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.513185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.513648 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.513648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.513698 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.514192 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.514196 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.514706 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.516412 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.516413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.516413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.516586 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.516806 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.516806 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.516807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.516891 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.517176 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.517177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.517213 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.517265 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.517343 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":75,"threads_started":1}
I20260808 17:45:55.517463 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.511160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.29.62.254:42553)
0808 17:45:55.511212 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.511215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.511277 (+ 62us) server_negotiation.cc:407] Connection header received
0808 17:45:55.511356 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.511357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.511367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.511429 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.511720 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.512517 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.513267 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.513570 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.513643 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.514800 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.514802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.514802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.514810 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.516334 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.516658 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.516659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.516659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.516740 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.516968 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.516969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.516969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.517110 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.517181 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.517292 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.517369 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:45:55.517727 6967 tablet_copy_service.cc:140] P ef37bce3589b40349d7954d1bb59e228: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bac033c9a915486d8910ac4bbc5636dd ({username='oryx', principal='oryx/127.29.62.253@KRBTEST.COM'} at 127.0.0.1:45514)
I20260808 17:45:55.517838 6967 tablet_copy_service.cc:161] P ef37bce3589b40349d7954d1bb59e228: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bac033c9a915486d8910ac4bbc5636dd at {username='oryx', principal='oryx/127.29.62.253@KRBTEST.COM'} at 127.0.0.1:45514: session id = bac033c9a915486d8910ac4bbc5636dd-00000000000000000000000000000000
I20260808 17:45:55.518563 6967 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 17:45:55.519215 7688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 17:45:55.520160 7688 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: tablet copy: Starting download of 0 data blocks...
I20260808 17:45:55.520314 7688 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: tablet copy: Starting download of 1 WAL segments...
I20260808 17:45:55.521234 7688 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 17:45:55.522037 6967 tablet_copy_service.cc:342] P ef37bce3589b40349d7954d1bb59e228: Request end of tablet copy session bac033c9a915486d8910ac4bbc5636dd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.62.253@KRBTEST.COM'} at 127.0.0.1:45514
I20260808 17:45:55.522099 6967 tablet_copy_service.cc:433] P ef37bce3589b40349d7954d1bb59e228: ending tablet copy session bac033c9a915486d8910ac4bbc5636dd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bac033c9a915486d8910ac4bbc5636dd
I20260808 17:45:55.524708 7688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:55.525749 7856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.525804 7852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.525877 7853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.526139 7688 server_base.cc:1034] running on GCE node
I20260808 17:45:55.526233 7688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:55.526363 7688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:55.527514 7688 hybrid_clock.cc:648] HybridClock initialized: now 1786211155527503 us; error 28 us; skew 500 ppm
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:55.528808 7688 init.cc:377] Logged in from keytab as oryx/127.29.62.253@KRBTEST.COM (short username oryx)
I20260808 17:45:55.529081 7688 webserver.cc:460] Webserver started at http://127.29.62.253:45681/ using document root <none> and password file <none>
I20260808 17:45:55.529213 7688 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.529269 7688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.529878 7688 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:55.530467 7868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.530591 7688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.530640 7688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
uuid: "bac033c9a915486d8910ac4bbc5636dd"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "38b9724c5d9a1d5e2ccd0c5992b6787a"
server_key_iv: "6a7fdf2ed45b7c27a1bf5987297d02a0"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.530699 7688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.544358 7688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:55.546176 7688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:55.549506 7688 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.253:44951
I20260808 17:45:55.549703 7948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.253:44951 every 8 connection(s)
I20260808 17:45:55.549969 7688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
I20260808 17:45:55.550812 7950 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:45:55.551259 7950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Bootstrap starting.
I20260808 17:45:55.553484 7950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:45:55.553730 7950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Bootstrap complete.
I20260808 17:45:55.553936 7950 raft_consensus.cc:348] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } }
I20260808 17:45:55.554015 7950 raft_consensus.cc:729] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bac033c9a915486d8910ac4bbc5636dd, State: Initialized, Role: LEARNER
I20260808 17:45:55.554044 7950 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } }
I20260808 17:45:55.554226 7950 sys_catalog.cc:564] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:55.554230 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } }
I20260808 17:45:55.554359 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: This master's current role is: LEARNER
I20260808 17:45:55.555833 7971 catalog_manager.cc:1259] Loaded cluster ID: 11c55d0c6a7040a39622e80b0dfcf358
I20260808 17:45:55.555924 7971 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: loading cluster ID for follower catalog manager: success
I20260808 17:45:55.556905 7971 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: acquiring CA information for follower catalog manager: success
I20260808 17:45:55.557392 7971 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:45:55.562705 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.555111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.29.62.253:44951)
0808 17:45:55.555571 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.555573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.555584 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:45:55.555609 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.555610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.555621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.555705 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.556464 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.557470 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.558221 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.558539 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.558616 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.559840 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.559842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.559842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.559849 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.561472 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.561818 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.561819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.561819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.561888 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.562109 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.562110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.562110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.562248 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.562361 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.562512 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.562612 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for oryx/127.29.62.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.252.keytab.
Entry for principal oryx/127.29.62.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.252.keytab.
I20260808 17:45:55.589147 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.252:46541
--webserver_interface=127.29.62.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.252.keytab
--principal=oryx/127.29.62.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:42553,127.29.62.253:44951,127.29.62.252:46541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:55.675796 7975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:55.676043 7975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:55.676080 7975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:55.677229 7975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:55.677302 7975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:55.677330 7975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:55.677358 7975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:55.677390 7975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:55.678962 7975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:42553,127.29.62.253:44951,127.29.62.252:46541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.252.keytab
--principal=oryx/127.29.62.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.252:46541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.62.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:55.679250 7975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:55.679497 7975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:55.684293 7982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.684352 7983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.684481 7975 server_base.cc:1034] running on GCE node
W20260808 17:45:55.684322 7987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.684844 7975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:55.685186 7975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:55.686327 7975 hybrid_clock.cc:648] HybridClock initialized: now 1786211155686309 us; error 41 us; skew 500 ppm
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:55.688970 7975 init.cc:377] Logged in from keytab as oryx/127.29.62.252@KRBTEST.COM (short username oryx)
I20260808 17:45:55.689394 7975 webserver.cc:460] Webserver started at http://127.29.62.252:39079/ using document root <none> and password file <none>
I20260808 17:45:55.689615 7975 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.689657 7975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.689749 7975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:55.690909 7975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data/instance:
uuid: "f750d1c0045a4c9a93ed3cb852accb33"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b9eead2ddb11861e8297baed728594f5"
server_key_iv: "64dc1e4bb64aa5b47ac77db697b36158"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.691239 7975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal/instance:
uuid: "f750d1c0045a4c9a93ed3cb852accb33"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b9eead2ddb11861e8297baed728594f5"
server_key_iv: "64dc1e4bb64aa5b47ac77db697b36158"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.692554 7975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 17:45:55.693233 7998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.693413 7975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.693529 7975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
uuid: "f750d1c0045a4c9a93ed3cb852accb33"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b9eead2ddb11861e8297baed728594f5"
server_key_iv: "64dc1e4bb64aa5b47ac77db697b36158"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.693614 7975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.705097 7975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:55.707181 7975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:55.707351 7975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:55.711687 7975 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.252:46541
I20260808 17:45:55.711715 8092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.252:46541 every 8 connection(s)
I20260808 17:45:55.712152 7975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
I20260808 17:45:55.713201 8093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:55.715068 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 7975
I20260808 17:45:55.714813 8093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.252@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.62.252@KRBTEST.COM
I20260808 17:45:55.724969 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.715725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.29.62.254:42553)
0808 17:45:55.715773 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.715775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.715984 (+ 209us) server_negotiation.cc:407] Connection header received
0808 17:45:55.716493 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.716495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.716508 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.716557 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.717084 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.717585 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.718770 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.719062 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.719133 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.721558 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.721559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.721559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.721567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.723474 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.723983 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.723984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.723985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.724053 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.724306 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.724307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.724307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.724442 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.724516 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.724793 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.724876 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:45:55.725163 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.715628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:45516)
0808 17:45:55.715847 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.715893 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.716384 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.716574 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.716577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.716829 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.716986 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.716990 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.717696 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.717697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.718634 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.718637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.719165 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.719166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.719243 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.719771 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.719782 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.721464 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.723573 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.723579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.723582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.723852 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.724130 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.724131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.724131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.724229 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.724519 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.724526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.724656 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.724758 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.724932 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":101,"threads_started":1}
I20260808 17:45:55.725793 8093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:45:55.725860 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.715770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41950 (local address 127.29.62.252:46541)
0808 17:45:55.715959 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.715963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.715988 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:45:55.716398 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.716409 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.716429 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.716530 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.717157 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.717953 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.718706 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.719198 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.719281 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.722122 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.722130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.722130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.722137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.724140 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.724464 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.724467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.724467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.724537 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.724756 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.724757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.724757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.724923 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.725153 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.725643 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.725725 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.252@KRBTEST.COM for oryx/127.29.62.253@KRBTEST.COM
I20260808 17:45:55.731947 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.726092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.0.0.1:51806)
0808 17:45:55.726142 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.726154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.726197 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.726310 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.726311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.726489 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.726526 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.726528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.727161 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.727163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.727656 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.727658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.728050 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.728051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.728111 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.728525 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.728530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.729644 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.731061 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.731062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.731062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.731221 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.731400 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.731411 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.731411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.731473 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.731734 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.731734 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.731767 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.731821 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.731888 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:45:55.731983 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.726154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51806 (local address 127.29.62.253:44951)
0808 17:45:55.726209 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.726211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.726221 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:55.726249 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.726250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.726260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.726310 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.726593 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.727078 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.727740 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.727968 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.728039 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.729723 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.729725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.729725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.729732 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.730991 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.731285 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.731286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.731286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.731335 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.731550 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.731551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.731552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.731678 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.731732 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.731856 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.731924 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:45:55.732395 8093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.252@KRBTEST.COM for oryx/127.29.62.252@KRBTEST.COM
I20260808 17:45:55.737691 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.732633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.0.0.1:41954)
0808 17:45:55.732678 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.732689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.732725 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.732820 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.732821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.732932 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.732957 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.732958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.733509 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.733509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.733946 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.733947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.734260 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.734261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.734295 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.734688 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.734693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.735713 (+ 1020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.736952 (+ 1239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.736953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.736953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.737105 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.737271 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.737271 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.737271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.737320 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.737485 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.737485 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.737516 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.737565 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.737635 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:45:55.737753 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.732694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41954 (local address 127.29.62.252:46541)
0808 17:45:55.732726 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.732728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.732736 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:45:55.732764 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.732764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.732773 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.732818 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.733013 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.733438 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.734015 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.734201 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.734249 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.735789 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.735790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.735790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.735796 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.736885 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.737168 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.737168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.737169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.737216 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.737357 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.737358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.737358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.737439 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.737488 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.737578 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.737680 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":16}
I20260808 17:45:55.739132 8093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Bootstrap starting.
I20260808 17:45:55.739993 8093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:55.740283 8093 log.cc:824] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:55.740995 8093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: No bootstrap required, opened a new log
I20260808 17:45:55.742460 8093 raft_consensus.cc:348] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } }
I20260808 17:45:55.742602 8093 raft_consensus.cc:374] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:55.742648 8093 raft_consensus.cc:729] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f750d1c0045a4c9a93ed3cb852accb33, State: Initialized, Role: FOLLOWER
I20260808 17:45:55.742791 8093 consensus_queue.cc:260] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } }
I20260808 17:45:55.743083 8093 sys_catalog.cc:564] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:55.743072 8098 sys_catalog.cc:455] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } } }
I20260808 17:45:55.743192 8098 sys_catalog.cc:458] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:45:55.744761 8114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:45:55.744817 8114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:45:55.746994 7975 master_runner.cc:428] Detected that this master f750d1c0045a4c9a93ed3cb852accb33 is joining an existing cluster
I20260808 17:45:55.747063 7975 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.62.252:46541
I20260808 17:45:55.773766 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.748171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.29.62.252:46541)
0808 17:45:55.748215 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.748217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.748507 (+ 290us) server_negotiation.cc:407] Connection header received
0808 17:45:55.748606 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.748607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.748617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.748676 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.749153 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.749876 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.750950 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.751205 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.751242 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.752337 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.752339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.752339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.752346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.772450 (+ 20104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.772880 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.772882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.772882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.772939 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.773176 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.773177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.773177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.773336 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.773414 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.773581 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.773665 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:45:55.774085 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.748006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.0.0.1:41966)
0808 17:45:55.748441 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.748450 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.748528 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.748675 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.748676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.748963 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.749070 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.749072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.749956 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.749956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.750864 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.750867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.751304 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.751304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.751373 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.751904 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.751908 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.752219 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.772543 (+ 20324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.772544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.772545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.772789 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.773028 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.773029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.773029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.773091 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.773438 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.773438 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.773475 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.773889 (+ 414us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.773981 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260808 17:45:55.775600 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.748187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.29.62.253:44951)
0808 17:45:55.748235 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.748237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.748404 (+ 167us) server_negotiation.cc:407] Connection header received
0808 17:45:55.748554 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.748556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.748568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.748600 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.749161 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.749907 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.751165 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.751954 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.752002 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.772109 (+ 20107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.772111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.772111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.772121 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.774364 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.774755 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.774756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.774756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.774816 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.775020 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.775021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.775021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.775161 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.775225 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.775423 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.775508 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:45:55.775625 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.748024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.0.0.1:51816)
0808 17:45:55.748305 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.748344 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.748474 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.748666 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.748667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.748972 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.749067 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.749070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.750004 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.750005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.751060 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.751064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.752082 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.752083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.752146 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.752613 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.752619 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.771974 (+ 19355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.774483 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.774484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.774484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.774664 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.774889 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.774890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.774890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.774954 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.775253 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.775254 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.775297 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.775383 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.775488 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260808 17:45:55.775993 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.748338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45524 (local address 127.29.62.254:42553)
0808 17:45:55.748383 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.748411 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.748835 (+ 424us) server_negotiation.cc:407] Connection header received
0808 17:45:55.748933 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.748934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.748946 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.748988 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.749389 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.750104 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.751117 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.751406 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.751448 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.772112 (+ 20664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.772114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.772114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.772124 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.773938 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.775085 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.775087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.775087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.775142 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.775363 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.775364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.775365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.775514 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.775579 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.775811 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.775896 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:45:55.776029 8127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.748495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:45524)
0808 17:45:55.748776 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.748786 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.748861 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.749058 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.749058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.749244 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.749303 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.749304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.750189 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.750189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.751037 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.751038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.751604 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.751605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.751668 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.770922 (+ 19254us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.770928 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.771980 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.774767 (+ 2787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.774768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.774769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.774980 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.775217 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.775218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.775219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.775283 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.775610 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.775611 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.775653 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.775749 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.775842 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":63,"threads_started":1}
I20260808 17:45:55.777241 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 7042
I20260808 17:45:55.801924 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:42193
--local_ip_for_outbound_sockets=127.29.62.193
--tserver_master_addrs=127.29.62.254:42553
--webserver_port=34655
--webserver_interface=127.29.62.193
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab
--principal=oryx/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951,127.29.62.252:46541 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211154, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.254@KRBTEST.COM for oryx/127.29.62.252@KRBTEST.COM
I20260808 17:45:55.805742 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.798334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.29.62.252:46541)
0808 17:45:55.798393 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.798395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.798463 (+ 68us) server_negotiation.cc:407] Connection header received
0808 17:45:55.798552 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.798552 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.798562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.798614 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.798926 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.799541 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.800320 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.800594 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.800677 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.802817 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.802819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.802819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.802827 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.804472 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.804888 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.804889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.804890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.804952 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.805177 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.805178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.805179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.805311 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.805379 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.805562 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.805646 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:45:55.805795 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.798300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.0.0.1:41976)
0808 17:45:55.798357 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.798370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.798484 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.798617 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.798617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.798801 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.798847 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.798849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.799652 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.799653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.800231 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.800234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.800677 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.800679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.800758 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.801196 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.801202 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.802717 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.804567 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.804568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.804569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.804806 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.805030 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.805031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.805031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.805107 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.805384 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.805385 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.805438 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.805521 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.805639 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:45:55.806350 6927 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.62.252:46541: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33"
member_type: NON_VOTER
last_known_addr {
host: "127.29.62.252"
port: 46541
}
attrs {
promote: true
}
}
dest_uuid: "ef37bce3589b40349d7954d1bb59e228"
cas_config_opid_index: 5
I20260808 17:45:55.806690 6927 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } }
I20260808 17:45:55.807076 7833 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.808245 7833 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.62.252:46541
I20260808 17:45:55.808326 7833 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.808422 7833 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:55.808883 8051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:55.808938 7975 master.cc:537] Master@127.29.62.252:46541 shutting down...
I20260808 17:45:55.809829 8051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 FOLLOWER]: Refusing update from remote peer ef37bce3589b40349d7954d1bb59e228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 17:45:55.810134 7835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 17:45:55.811085 6907 consensus_peers.cc:487] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 -> Peer f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252:46541): Couldn't send request to peer f750d1c0045a4c9a93ed3cb852accb33. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 17:45:55.813536 7975 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:55.813640 7975 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:55.813707 7975 tablet_replica.cc:330] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: stopping tablet replica
I20260808 17:45:55.816587 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.808698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.0.0.1:51830)
0808 17:45:55.809109 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.809123 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.809170 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.812333 (+ 3163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.812334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.812336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:55.812380 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.812381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.814110 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.814111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.815368 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.815371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.816285 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.816286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.816384 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.816409 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.816467 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.816515 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390}
I20260808 17:45:55.816852 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.810019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.29.62.253:44951)
0808 17:45:55.811756 (+ 1737us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.811758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.811768 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:55.811795 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.811796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.811821 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.812466 (+ 645us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:55.812986 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.814004 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.815489 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.816085 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.816157 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.816229 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.816741 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.816790 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1707}
I20260808 17:45:55.817047 7906 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 LEARNER]: Refusing update from remote peer ef37bce3589b40349d7954d1bb59e228: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 17:45:55.817231 7835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 17:45:55.817757 7906 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.819126 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef37bce3589b40349d7954d1bb59e228. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.819216 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: This master's current role is: LEARNER
I20260808 17:45:55.819252 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: NON_VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: true } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.819296 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: This master's current role is: LEARNER
I20260808 17:45:55.819514 7835 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: attempting to promote NON_VOTER bac033c9a915486d8910ac4bbc5636dd to VOTER
I20260808 17:45:55.819700 7835 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } }
I20260808 17:45:55.820160 7906 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 LEARNER]: Refusing update from remote peer ef37bce3589b40349d7954d1bb59e228: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 17:45:55.820168 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.820059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.0.0.1:41980)
0808 17:45:55.820100 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.820139 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:46541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 17:45:55.820384 8137 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 17:45:55.820549 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.820443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.0.0.1:41986)
0808 17:45:55.820497 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.820520 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:46541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 17:45:55.820670 6907 consensus_peers.cc:487] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 -> Peer f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252:46541): Couldn't send request to peer f750d1c0045a4c9a93ed3cb852accb33. Status: Network error: Client connection negotiation failed: client connection to 127.29.62.252:46541: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 17:45:55.821209 7833 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bac033c9a915486d8910ac4bbc5636dd (127.29.62.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.822096 7833 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.822176 7833 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:55.822232 7833 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.822275 7833 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:55.822540 7906 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bac033c9a915486d8910ac4bbc5636dd (127.29.62.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.823563 8135 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.823663 8135 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:45:55.830269 7975 master.cc:552] Master@127.29.62.252:46541 shutdown complete.
I20260808 17:45:55.830322 7975 master_runner.cc:305] Clearing existing system tablet
I20260808 17:45:55.830488 7975 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.830546 7975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.831300 7975 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:55.831915 8139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.832073 7975 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.832136 7975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:55.832159 7975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
uuid: "f750d1c0045a4c9a93ed3cb852accb33"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b9eead2ddb11861e8297baed728594f5"
server_key_iv: "64dc1e4bb64aa5b47ac77db697b36158"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.832876 7975 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:55.834322 7975 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 17:45:55.834421 7975 log.cc:1196] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 17:45:55.834638 7975 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Deleting consensus metadata
I20260808 17:45:55.834764 7975 master_runner.cc:315] Copying system tablet from 127.29.62.254:42553
I20260808 17:45:55.835368 7975 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: tablet copy: Beginning tablet copy session from remote peer at address 127.29.62.254:42553
I20260808 17:45:55.842727 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.835717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45528 (local address 127.29.62.254:42553)
0808 17:45:55.835761 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.835763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.835822 (+ 59us) server_negotiation.cc:407] Connection header received
0808 17:45:55.835887 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.835888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.835898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.835964 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.836307 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.837069 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.838095 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.838405 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.838471 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.839678 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.839680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.839680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.839687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.841411 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.841816 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.841817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.841817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.841873 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.842080 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.842080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.842081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.842224 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.842265 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.842546 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.842623 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:45:55.843060 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.835655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:45528)
0808 17:45:55.835770 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.835780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.835837 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.835970 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.835971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.836169 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.836224 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.836227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.837167 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.837168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.838004 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.838007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.838556 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.838557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.838621 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.839149 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.839155 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.839572 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.841515 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.841517 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.841517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.841733 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.841941 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.841942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.841942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.841999 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.842310 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.842311 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.842355 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.842841 (+ 486us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.842957 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260808 17:45:55.843304 6967 tablet_copy_service.cc:140] P ef37bce3589b40349d7954d1bb59e228: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f750d1c0045a4c9a93ed3cb852accb33 ({username='oryx', principal='oryx/127.29.62.252@KRBTEST.COM'} at 127.0.0.1:45528)
I20260808 17:45:55.843358 6967 tablet_copy_service.cc:161] P ef37bce3589b40349d7954d1bb59e228: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f750d1c0045a4c9a93ed3cb852accb33 at {username='oryx', principal='oryx/127.29.62.252@KRBTEST.COM'} at 127.0.0.1:45528: session id = f750d1c0045a4c9a93ed3cb852accb33-00000000000000000000000000000000
I20260808 17:45:55.843940 6967 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 17:45:55.844470 7975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 17:45:55.845809 7975 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: tablet copy: Starting download of 0 data blocks...
I20260808 17:45:55.845952 7975 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: tablet copy: Starting download of 1 WAL segments...
I20260808 17:45:55.846762 7975 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 17:45:55.847680 6967 tablet_copy_service.cc:342] P ef37bce3589b40349d7954d1bb59e228: Request end of tablet copy session f750d1c0045a4c9a93ed3cb852accb33-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.62.252@KRBTEST.COM'} at 127.0.0.1:45528
I20260808 17:45:55.847755 6967 tablet_copy_service.cc:433] P ef37bce3589b40349d7954d1bb59e228: ending tablet copy session f750d1c0045a4c9a93ed3cb852accb33-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f750d1c0045a4c9a93ed3cb852accb33
I20260808 17:45:55.850826 7975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:55.851938 8159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.851949 8156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.858546 8161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.858836 7975 server_base.cc:1034] running on GCE node
I20260808 17:45:55.858943 7975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:55.859084 7975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:55.860231 7975 hybrid_clock.cc:648] HybridClock initialized: now 1786211155860217 us; error 29 us; skew 500 ppm
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:55.861817 7975 init.cc:377] Logged in from keytab as oryx/127.29.62.252@KRBTEST.COM (short username oryx)
I20260808 17:45:55.862107 7975 webserver.cc:460] Webserver started at http://127.29.62.252:40279/ using document root <none> and password file <none>
I20260808 17:45:55.862248 7975 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.862288 7975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.863134 7975 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:55.863803 8175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.863986 7975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:55.864050 7975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
uuid: "f750d1c0045a4c9a93ed3cb852accb33"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b9eead2ddb11861e8297baed728594f5"
server_key_iv: "64dc1e4bb64aa5b47ac77db697b36158"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.864123 7975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.889940 7975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:55.891933 7975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:55.895293 7975 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.252:46541
I20260808 17:45:55.895431 8258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.252:46541 every 8 connection(s)
I20260808 17:45:55.895663 7975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
I20260808 17:45:55.896672 8259 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:45:55.897230 8259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Bootstrap starting.
I20260808 17:45:55.900512 8259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:45:55.900736 8259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: Bootstrap complete.
I20260808 17:45:55.900943 8259 raft_consensus.cc:348] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } }
I20260808 17:45:55.901044 8259 raft_consensus.cc:729] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f750d1c0045a4c9a93ed3cb852accb33, State: Initialized, Role: LEARNER
I20260808 17:45:55.901087 8259 consensus_queue.cc:260] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } }
I20260808 17:45:55.901269 8259 sys_catalog.cc:564] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:55.901270 8263 sys_catalog.cc:455] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:55.901485 8263 sys_catalog.cc:458] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: This master's current role is: LEARNER
I20260808 17:45:55.903764 8282 catalog_manager.cc:1259] Loaded cluster ID: 11c55d0c6a7040a39622e80b0dfcf358
I20260808 17:45:55.903813 8282 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: loading cluster ID for follower catalog manager: success
I20260808 17:45:55.904933 8282 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: acquiring CA information for follower catalog manager: success
I20260808 17:45:55.905217 8282 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 17:45:55.907671 8132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:55.907888 8132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:55.907938 8132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:55.909121 8132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:55.909185 8132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:55.909230 8132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:45:55.910818 8132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab
--principal=oryx/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:42193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=34655
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951,127.29.62.252:46541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:55.911077 8132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:55.911303 8132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:55.916213 8290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.916241 8293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:55.916175 8289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:55.916785 8132 server_base.cc:1034] running on GCE node
I20260808 17:45:55.916988 8132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:55.917237 8132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:55.918463 8132 hybrid_clock.cc:648] HybridClock initialized: now 1786211155918438 us; error 33 us; skew 500 ppm
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:55.920914 8132 init.cc:377] Logged in from keytab as oryx/127.29.62.193@KRBTEST.COM (short username oryx)
I20260808 17:45:55.921363 8132 webserver.cc:460] Webserver started at http://127.29.62.193:34655/ using document root <none> and password file <none>
I20260808 17:45:55.921666 8132 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:55.921753 8132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:55.923328 8132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:55.923939 8305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.924120 8132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:55.924230 8132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "b9c78ccb5ccf415c96be50fd36c64f17"
format_stamp: "Formatted at 2026-08-08 17:45:54 on dist-test-slave-tk9z"
server_key: "65be4391220e81490333f31c2bf3a225"
server_key_iv: "dd9729a19962f39655fb4235f94d2cfe"
server_key_version: "encryptionkey@0"
I20260808 17:45:55.924610 8132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:55.946834 8132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:55.948980 8132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:55.949162 8132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:55.949446 8132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:55.949785 8132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:55.949841 8132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.949882 8132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:55.949911 8132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:55.956336 8132 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:42193
I20260808 17:45:55.956391 8494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:42193 every 8 connection(s)
I20260808 17:45:55.956763 8132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:45:55.959277 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 8132
I20260808 17:45:55.959457 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 7257
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.252@KRBTEST.COM
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.253@KRBTEST.COM
I20260808 17:45:55.970199 8502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.957471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.29.62.193:39887)
0808 17:45:55.957735 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.957786 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.959070 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.959377 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.959382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.959829 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.960082 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.960087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.961146 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.961147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.962231 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.962234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.962956 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.962957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.963155 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.963778 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.963788 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.965603 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.967826 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.967834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.967836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.968241 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.968516 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.968518 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.968518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.968800 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.969190 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.969198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.969441 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.969596 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.969827 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":100,"threads_started":1}
I20260808 17:45:55.970467 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.957470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:59459 (local address 127.29.62.253:44951)
0808 17:45:55.957526 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.957528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.957995 (+ 467us) server_negotiation.cc:407] Connection header received
0808 17:45:55.959205 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.959207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.959221 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.959256 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.962229 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.962925 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.964475 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.964899 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.964970 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.967749 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.967751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.967751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.967758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.969256 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.969602 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.969602 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.969603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.969648 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.969812 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.969813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.969813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.970007 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.970047 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.970223 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.970322 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:45:55.970745 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.957502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:39887 (local address 127.29.62.252:46541)
0808 17:45:55.957624 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.957627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.958441 (+ 814us) server_negotiation.cc:407] Connection header received
0808 17:45:55.959184 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.959185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.959201 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.959281 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.960182 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.961055 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.962321 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.962791 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.962872 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.965723 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.965725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.965725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.965735 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.967719 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.968340 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.968342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.968342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.968431 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.968890 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.968891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.968891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.969077 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.969155 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.970464 (+ 1309us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.970588 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260808 17:45:55.971306 8504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.957471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.29.62.193:59459)
0808 17:45:55.957914 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.957927 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.959085 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.959377 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.959383 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.961984 (+ 2601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.962116 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.962119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.963021 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.963022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.964355 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.964359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.965249 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.965250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.965339 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.965771 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.965776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.967659 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.969343 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.969343 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.969344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.969536 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.969705 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.969705 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.969706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.969757 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.970091 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.970092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.970131 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.971051 (+ 920us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.971162 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":100,"threads_started":1}
I20260808 17:45:55.971992 8497 heartbeater.cc:347] Connected to a master server at 127.29.62.253:44951
I20260808 17:45:55.972133 8497 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:55.972419 8497 heartbeater.cc:511] Master 127.29.62.253:44951 requested a full tablet report, sending...
I20260808 17:45:55.973078 7891 ts_manager.cc:194] Registered new tserver with Master: b9c78ccb5ccf415c96be50fd36c64f17 (127.29.62.193:42193)
I20260808 17:45:55.975844 8495 heartbeater.cc:347] Connected to a master server at 127.29.62.252:46541
I20260808 17:45:55.975950 8495 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:55.976074 8495 heartbeater.cc:511] Master 127.29.62.252:46541 requested a full tablet report, sending...
I20260808 17:45:55.976480 8199 ts_manager.cc:194] Registered new tserver with Master: b9c78ccb5ccf415c96be50fd36c64f17 (127.29.62.193:42193)
Aug 08 17:45:55 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211155, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:45:55.979614 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:44579
--local_ip_for_outbound_sockets=127.29.62.194
--tserver_master_addrs=127.29.62.254:42553
--webserver_port=32851
--webserver_interface=127.29.62.194
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab
--principal=oryx/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951,127.29.62.252:46541 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:45:55.981937 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.957886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:57677 (local address 127.29.62.254:42553)
0808 17:45:55.957929 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:55.957931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:55.958018 (+ 87us) server_negotiation.cc:407] Connection header received
0808 17:45:55.974977 (+ 16959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:55.974979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:55.974993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:55.975076 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:55.975386 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.976067 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.976982 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.977177 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.977266 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.979259 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:55.979260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:55.979260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:55.979267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:55.980780 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.981140 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.981141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.981142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.981206 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:55.981405 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:55.981406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:55.981407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:55.981527 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:55.981601 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:55.981763 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 17:45:55.981843 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:45:55.981961 8501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:55.957780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.193:57677)
0808 17:45:55.957930 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:55.957938 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:55.974880 (+ 16942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:55.975084 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:55.975085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:55.975238 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:55.975292 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.975294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.976165 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.976166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.976887 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:55.976889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:55.977267 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:55.977268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:55.977334 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:55.977799 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:55.977804 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:55.979078 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:55.980884 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.980885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.980885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.981064 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.981274 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:55.981275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:55.981276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:55.981333 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:55.981605 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:55.981606 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:55.981649 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 17:45:55.981736 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 17:45:55.981843 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260808 17:45:55.982259 8496 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42553
I20260808 17:45:55.982331 8496 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:55.982429 8496 heartbeater.cc:511] Master 127.29.62.254:42553 requested a full tablet report, sending...
I20260808 17:45:55.982702 6927 ts_manager.cc:194] Re-registered known tserver with Master: b9c78ccb5ccf415c96be50fd36c64f17 (127.29.62.193:42193)
I20260808 17:45:55.983103 6927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:57677
W20260808 17:45:56.065416 8505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:56.065644 8505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:56.065685 8505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:56.066913 8505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:56.066973 8505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:56.067020 8505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:45:56.068542 8505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab
--principal=oryx/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:44579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=32851
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951,127.29.62.252:46541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:56.068828 8505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:56.069077 8505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:56.074110 8519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:56.074187 8505 server_base.cc:1034] running on GCE node
W20260808 17:45:56.074096 8515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:56.074105 8514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:56.074553 8505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:56.074802 8505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:56.075941 8505 hybrid_clock.cc:648] HybridClock initialized: now 1786211156075921 us; error 37 us; skew 500 ppm
Aug 08 17:45:56 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:56.078466 8505 init.cc:377] Logged in from keytab as oryx/127.29.62.194@KRBTEST.COM (short username oryx)
I20260808 17:45:56.078892 8505 webserver.cc:460] Webserver started at http://127.29.62.194:32851/ using document root <none> and password file <none>
I20260808 17:45:56.079131 8505 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:56.079219 8505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:56.080567 8505 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:56.081216 8537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:56.081393 8505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:45:56.081509 8505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "cb4adc4acd0546f79732e4e9430c2484"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "c8b09523c1a9f9a2261729588f1d3ed0"
server_key_iv: "f5ec005443990bd242f92d0d86cea002"
server_key_version: "encryptionkey@0"
I20260808 17:45:56.081862 8505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:56.094099 8505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:56.096109 8505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:56.096279 8505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:56.096525 8505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:56.096853 8505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:56.096921 8505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:56.096967 8505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:56.096999 8505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:56.103399 8505 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:44579
I20260808 17:45:56.103446 8722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:44579 every 8 connection(s)
I20260808 17:45:56.103799 8505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:45:56.105304 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 8505
I20260808 17:45:56.105486 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 7475
Aug 08 17:45:56 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.253@KRBTEST.COM
I20260808 17:45:56.116945 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.104557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:46735 (local address 127.29.62.253:44951)
0808 17:45:56.104629 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.104631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.104895 (+ 264us) server_negotiation.cc:407] Connection header received
0808 17:45:56.106261 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.106262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.106276 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.106304 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.107199 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.107835 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.109184 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.109459 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.109496 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.112191 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.112193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.112193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.112200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.114832 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.115493 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.115495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.115496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.115567 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.115949 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.115951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.115951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.116134 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.116188 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.116667 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.116775 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:45:56.117689 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.104570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.29.62.194:46735)
0808 17:45:56.104779 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.104815 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.106147 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.106487 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.106490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.106842 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:56.107104 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.107109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.107944 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.107945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.109050 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.109052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.109577 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.109577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.109712 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.110339 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:56.110347 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:56.112094 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:56.114950 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.114959 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.114962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.115386 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.115656 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.115657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.115657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.115833 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.116248 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:56.116256 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:56.116463 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.117101 (+ 638us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.117338 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"spinlock_wait_cycles":8576,"thread_start_us":107,"threads_started":1}
Aug 08 17:45:56 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:45:56.125818 8726 heartbeater.cc:347] Connected to a master server at 127.29.62.253:44951
I20260808 17:45:56.125931 8726 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:56.126163 8726 heartbeater.cc:511] Master 127.29.62.253:44951 requested a full tablet report, sending...
I20260808 17:45:56.127662 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.105067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:56791 (local address 127.29.62.254:42553)
0808 17:45:56.105123 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.105126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.118141 (+ 13015us) server_negotiation.cc:407] Connection header received
0808 17:45:56.118273 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.118274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.118291 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.118327 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.118819 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.119818 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.121157 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.121549 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.121604 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.124511 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.124512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.124513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.124522 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.126460 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.126868 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.126869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.126869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.126928 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.127131 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.127131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.127132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.127298 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.127349 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.127530 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.127599 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:45:56.127737 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.104941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.194:56791)
0808 17:45:56.118076 (+ 13135us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.118088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.118175 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.118424 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.118425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.118635 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:56.118705 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.118707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.119941 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.119942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.121034 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.121038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.121697 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.121698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.121795 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.122418 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:56.122424 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:56.124382 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:56.126561 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.126562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.126562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.126793 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.126994 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.126994 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.126995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.127068 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.127364 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:56.127365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:56.127415 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.127481 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.127591 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13120,"thread_start_us":78,"threads_started":1}
I20260808 17:45:56.128101 8724 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42553
I20260808 17:45:56.128160 8724 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:56.128240 8724 heartbeater.cc:511] Master 127.29.62.254:42553 requested a full tablet report, sending...
I20260808 17:45:56.128507 6927 ts_manager.cc:194] Re-registered known tserver with Master: cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579)
I20260808 17:45:56.128888 6927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:56791
I20260808 17:45:56.130679 7891 ts_manager.cc:194] Registered new tserver with Master: cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579)
I20260808 17:45:56.131294 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:45125
--local_ip_for_outbound_sockets=127.29.62.195
--tserver_master_addrs=127.29.62.254:42553
--webserver_port=37859
--webserver_interface=127.29.62.195
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab
--principal=oryx/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951,127.29.62.252:46541 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:45:56 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.252@KRBTEST.COM
I20260808 17:45:56.138008 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.104640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:51371 (local address 127.29.62.252:46541)
0808 17:45:56.104681 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.104683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.105152 (+ 469us) server_negotiation.cc:407] Connection header received
0808 17:45:56.131102 (+ 25950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.131104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.131123 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.131200 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.131647 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.132140 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.133086 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.133296 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.133356 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.135343 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.135345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.135345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.135354 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.136814 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.137184 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.137186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.137186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.137251 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.137463 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.137464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.137464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.137580 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.137654 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.137821 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.137913 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:45:56.138019 8734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.104570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.29.62.194:51371)
0808 17:45:56.105086 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.105097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.130981 (+ 25884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.131204 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.131205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.131474 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:56.131548 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.131550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.132227 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.132228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.132992 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.132995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.133374 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.133375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.133457 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.133887 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:56.133892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:56.135246 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:56.136926 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.136927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.136928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.137106 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.137322 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.137322 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.137323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.137384 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.137656 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:56.137657 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:56.137701 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.137781 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.137898 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":24,"thread_start_us":102,"threads_started":1}
I20260808 17:45:56.138343 8723 heartbeater.cc:347] Connected to a master server at 127.29.62.252:46541
I20260808 17:45:56.138418 8723 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:56.138523 8723 heartbeater.cc:511] Master 127.29.62.252:46541 requested a full tablet report, sending...
I20260808 17:45:56.138846 8199 ts_manager.cc:194] Registered new tserver with Master: cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579)
W20260808 17:45:56.215885 8735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:56.216084 8735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:56.216106 8735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:56.217314 8735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:56.217357 8735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:56.217412 8735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:45:56.218978 8735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab
--principal=oryx/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:45125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=37859
--tserver_master_addrs=127.29.62.254:42553,127.29.62.253:44951,127.29.62.252:46541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:56.219169 8735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:56.219347 8735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:56.224373 8741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:56.224440 8745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:56.224668 8742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:56.224784 8735 server_base.cc:1034] running on GCE node
I20260808 17:45:56.224988 8735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:56.225236 8735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:56.226435 8735 hybrid_clock.cc:648] HybridClock initialized: now 1786211156226416 us; error 31 us; skew 500 ppm
I20260808 17:45:56.227391 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.223944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.29.62.252:46541)
0808 17:45:56.224037 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.224045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.224056 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:45:56.224096 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.224097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.224118 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.224210 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:56.224372 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.225134 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.226430 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.227061 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.227153 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.227229 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.227301 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.227336 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 17:45:56.227428 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.223798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.0.0.1:42000)
0808 17:45:56.223926 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.223937 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.223987 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.224231 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.224233 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.224235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:56.224301 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.224304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.225229 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.225231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.226293 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.226296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.227147 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.227148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.227196 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.227200 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.227292 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.227341 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 17:45:56.227773 8220 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 17:45:56.227955 8263 sys_catalog.cc:455] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef37bce3589b40349d7954d1bb59e228. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:56.228077 8263 sys_catalog.cc:458] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: This master's current role is: LEARNER
Aug 08 17:45:56 dist-test-slave-tk9z krb5kdc[6854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:56.229476 8735 init.cc:377] Logged in from keytab as oryx/127.29.62.195@KRBTEST.COM (short username oryx)
I20260808 17:45:56.229919 8735 webserver.cc:460] Webserver started at http://127.29.62.195:37859/ using document root <none> and password file <none>
I20260808 17:45:56.230155 8735 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:56.230211 8735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:56.231649 8735 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:56.232281 8756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:56.232470 8735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:56.232579 8735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "be99fa38df004be29e01f1dd07c81b41"
format_stamp: "Formatted at 2026-08-08 17:45:55 on dist-test-slave-tk9z"
server_key: "b50f2df6eb75fef32f179ae73d1ebc1d"
server_key_iv: "3db3c1d3e0534b7ec3d76979ec8f92c8"
server_key_version: "encryptionkey@0"
I20260808 17:45:56.232947 8735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:56.244218 8735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:56.246001 8735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:56.246160 8735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:56.246411 8735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:45:56.246734 8735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:45:56.246788 8735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:56.246830 8735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:45:56.246857 8735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:56.252637 8735 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:45125
I20260808 17:45:56.252691 8955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:45125 every 8 connection(s)
I20260808 17:45:56.253021 8735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:45:56.257191 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 8735
Aug 08 17:45:56 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
Aug 08 17:45:56 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.253@KRBTEST.COM
Aug 08 17:45:56 dist-test-slave-tk9z krb5kdc[6854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211156, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.252@KRBTEST.COM
I20260808 17:45:56.266651 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.254203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:52309 (local address 127.29.62.254:42553)
0808 17:45:56.254254 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.254256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.254265 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:56.255350 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.255352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.255383 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.255414 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.256271 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.257021 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258365 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.258678 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258721 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.263000 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.263001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.263001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.263008 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.265374 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.265761 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.265762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.265763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.265819 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.266054 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.266055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.266056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.266202 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.266270 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.266455 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.266535 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:45:56.266682 8966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.253998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.195:52309)
0808 17:45:56.254131 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.254140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.255233 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.255506 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.255511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.255936 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:56.256158 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.256164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.257121 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.257122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.258256 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.258258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.258804 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.258873 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.259666 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:56.259676 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:56.262895 (+ 3219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:56.265453 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.265454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.265455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.265677 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.265900 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.265912 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.265913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.265975 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.266263 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:56.266264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:56.266302 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.266407 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.266499 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":94,"thread_start_us":91,"threads_started":1}
I20260808 17:45:56.266719 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.253773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:43037 (local address 127.29.62.253:44951)
0808 17:45:56.253823 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.253825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.254024 (+ 199us) server_negotiation.cc:407] Connection header received
0808 17:45:56.255351 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.255352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.255380 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.255414 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.256254 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.256753 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258364 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.258606 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258691 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.262568 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.262569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.262569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.262575 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.264495 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.265201 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.265203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.265203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.265266 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.265657 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.265659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.265659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.265798 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.265861 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.266557 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.266628 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:45:56.266820 8964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.253689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.29.62.195:43037)
0808 17:45:56.253862 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.253904 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.255228 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.255506 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.255511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.255951 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:56.256158 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.256163 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.256863 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.256864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.258259 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.258263 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.258700 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.258850 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.259788 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:56.259793 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:56.262468 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:56.264560 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.264567 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.264569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.264870 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.265386 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.265387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.265387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.265572 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.265890 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:56.265897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:56.266082 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.266197 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.266384 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260808 17:45:56.267275 8960 heartbeater.cc:347] Connected to a master server at 127.29.62.253:44951
I20260808 17:45:56.267372 8960 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:56.267570 8960 heartbeater.cc:511] Master 127.29.62.253:44951 requested a full tablet report, sending...
I20260808 17:45:56.268090 8958 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42553
I20260808 17:45:56.268368 8958 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:56.268473 8958 heartbeater.cc:511] Master 127.29.62.254:42553 requested a full tablet report, sending...
I20260808 17:45:56.269023 7891 ts_manager.cc:194] Registered new tserver with Master: be99fa38df004be29e01f1dd07c81b41 (127.29.62.195:45125)
I20260808 17:45:56.269291 6927 ts_manager.cc:194] Re-registered known tserver with Master: be99fa38df004be29e01f1dd07c81b41 (127.29.62.195:45125)
I20260808 17:45:56.269759 6927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:52309
I20260808 17:45:56.271114 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.253685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:58053 (local address 127.29.62.252:46541)
0808 17:45:56.253740 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.253743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.254000 (+ 257us) server_negotiation.cc:407] Connection header received
0808 17:45:56.255336 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.255338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.255355 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.255400 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.256272 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.257027 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258405 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.258702 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258739 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.267925 (+ 9186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.267927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.267927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.267935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.269600 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.270002 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.270003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.270004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.270067 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.270437 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.270438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.270438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.270604 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.270674 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.270872 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.271002 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:45:56.271481 8963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.253690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.29.62.195:58053)
0808 17:45:56.253875 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.253910 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.255228 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.255506 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.255510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.255935 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:45:56.256158 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.256164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.257121 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.257122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.258256 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.258258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.258808 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.258809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.258873 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.259667 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:45:56.259676 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:45:56.267855 (+ 8179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:45:56.269686 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.269687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.269687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.269899 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.270140 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:45:56.270140 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:45:56.270141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:45:56.270188 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:45:56.270705 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:45:56.270706 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:45:56.270749 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.270972 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.271357 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":43,"thread_start_us":97,"threads_started":1}
I20260808 17:45:56.272066 8957 heartbeater.cc:347] Connected to a master server at 127.29.62.252:46541
I20260808 17:45:56.272127 8957 heartbeater.cc:464] Registering TS with master...
I20260808 17:45:56.272199 8957 heartbeater.cc:511] Master 127.29.62.252:46541 requested a full tablet report, sending...
I20260808 17:45:56.272431 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.258227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.29.62.254:42553)
0808 17:45:56.259341 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.259343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.259418 (+ 75us) server_negotiation.cc:407] Connection header received
0808 17:45:56.259518 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.259519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.259532 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.259597 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.261482 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.262258 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.263313 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.263655 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.263717 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.265365 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.265366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.265366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.265372 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.269609 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.270783 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.270784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.270784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.270854 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.271065 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.271066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.271066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.271417 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.271739 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.272205 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.272346 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1082,"thread_start_us":91,"threads_started":1}
I20260808 17:45:56.272490 8199 ts_manager.cc:194] Registered new tserver with Master: be99fa38df004be29e01f1dd07c81b41 (127.29.62.195:45125)
I20260808 17:45:56.273056 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.259075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.29.62.252:46541)
0808 17:45:56.259199 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.259200 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.259452 (+ 252us) server_negotiation.cc:407] Connection header received
0808 17:45:56.259579 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.259580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.259901 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.259976 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.260996 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.261847 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.267215 (+ 5368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.267499 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.267572 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.268813 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.268815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.268816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.268823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.271294 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.271694 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.271695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.271696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.271753 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.271978 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.271979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.271979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.272112 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.272567 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.272606 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.272693 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260808 17:45:56.273250 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.259864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.29.62.253:44951)
0808 17:45:56.261139 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.261141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.261148 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:45:56.261167 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.261168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.261178 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.261235 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.261938 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.262657 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.263747 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.264070 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.264136 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.265967 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.265968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.265969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.265974 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.271924 (+ 5950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.272283 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.272284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.272284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.272341 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.272560 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.272561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.272561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.272698 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.272794 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.272961 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.273047 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":75,"threads_started":1}
I20260808 17:45:56.274326 6927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:56.275923 6927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:56.276721 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: NON_VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: true } } }
I20260808 17:45:56.276835 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:56.278244 7833 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: attempting to promote NON_VOTER f750d1c0045a4c9a93ed3cb852accb33 to VOTER
I20260808 17:45:56.278403 7833 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } }
I20260808 17:45:56.278791 7906 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Refusing update from remote peer ef37bce3589b40349d7954d1bb59e228: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 17:45:56.279089 8987 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 17:45:56.279211 8218 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 LEARNER]: Refusing update from remote peer ef37bce3589b40349d7954d1bb59e228: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 17:45:56.279439 8987 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 17:45:56.279752 8987 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:56.279999 7906 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:56.280226 8218 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:56.280675 8987 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:56.280779 8987 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:56.280836 8987 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:56.280896 8987 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:56.280958 8987 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:56.281006 8987 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:56.281420 8263 sys_catalog.cc:455] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:56.281512 8263 sys_catalog.cc:458] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:45:56.281469 8135 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef37bce3589b40349d7954d1bb59e228" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:56.281553 8135 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:45:56.286489 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.282183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.29.62.193:42193)
0808 17:45:56.282332 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.282337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.282538 (+ 201us) server_negotiation.cc:407] Connection header received
0808 17:45:56.282587 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.282589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.282605 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.282686 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:56.283023 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.284152 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.285605 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.286101 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.286194 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.286279 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.286347 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.286406 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260808 17:45:56.286552 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.281976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:44579 (local address 127.0.0.1:33074)
0808 17:45:56.282030 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.282041 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.282079 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.283016 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.283017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.283018 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:56.283054 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.283055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.284581 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.284582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.285368 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.285370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.286228 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.286229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.286290 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.286294 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.286343 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.286441 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 17:45:56.286489 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.281982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42193 (local address 127.0.0.1:40994)
0808 17:45:56.282451 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.282461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.282521 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.282866 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.282867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.282870 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:56.282916 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.282917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.284247 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.284248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.285340 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.285342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.286194 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.286195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.286259 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.286265 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.286332 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.286395 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":337,"thread_start_us":87,"threads_started":1}
I20260808 17:45:56.287146 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.282077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.29.62.194:44579)
0808 17:45:56.282842 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.282846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.282856 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:56.282896 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.282898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.282915 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.283241 (+ 326us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:56.283305 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.284498 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.285654 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.286138 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.286229 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.286285 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.286533 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.287048 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":92,"threads_started":1}
I20260808 17:45:56.287344 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.282043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45125 (local address 127.0.0.1:55734)
0808 17:45:56.282165 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.282176 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.282230 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.282677 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.282678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.282681 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:56.282733 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.282734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.284075 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.284076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.285325 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.285328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.287100 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.287102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.287166 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.287170 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.287225 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.287269 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260808 17:45:56.287333 8391 tablet_service.cc:1467] Processing CreateTablet for tablet 14c5edf24ac84541bf72358472083459 (DEFAULT_TABLE table=test-table [id=d0adcd226b6e4b979de2af2938853d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:56.287598 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.282239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55734 (local address 127.29.62.195:45125)
0808 17:45:56.282397 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.282402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.282412 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:56.282451 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.282454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.282470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.282820 (+ 350us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:56.282879 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.283983 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.285477 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.286919 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.286996 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.287066 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.287508 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.287541 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260808 17:45:56.287734 8391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14c5edf24ac84541bf72358472083459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:56.287837 8609 tablet_service.cc:1467] Processing CreateTablet for tablet 14c5edf24ac84541bf72358472083459 (DEFAULT_TABLE table=test-table [id=d0adcd226b6e4b979de2af2938853d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:56.288156 8609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14c5edf24ac84541bf72358472083459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:56.288226 8835 tablet_service.cc:1467] Processing CreateTablet for tablet 14c5edf24ac84541bf72358472083459 (DEFAULT_TABLE table=test-table [id=d0adcd226b6e4b979de2af2938853d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:56.288506 8835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14c5edf24ac84541bf72358472083459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:56.291201 9006 tablet_bootstrap.cc:492] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Bootstrap starting.
I20260808 17:45:56.291896 9006 tablet_bootstrap.cc:654] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:56.292085 9011 tablet_bootstrap.cc:492] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Bootstrap starting.
I20260808 17:45:56.292199 9009 tablet_bootstrap.cc:492] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Bootstrap starting.
I20260808 17:45:56.292291 9006 log.cc:824] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:56.292795 9011 tablet_bootstrap.cc:654] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:56.292804 9009 tablet_bootstrap.cc:654] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:56.293068 9006 tablet_bootstrap.cc:492] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: No bootstrap required, opened a new log
I20260808 17:45:56.293148 9006 ts_tablet_manager.cc:1397] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:56.293144 9011 log.cc:824] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:56.293139 9009 log.cc:824] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:56.293915 9009 tablet_bootstrap.cc:492] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: No bootstrap required, opened a new log
I20260808 17:45:56.293987 9009 ts_tablet_manager.cc:1397] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:56.294169 9011 tablet_bootstrap.cc:492] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: No bootstrap required, opened a new log
I20260808 17:45:56.294235 9011 ts_tablet_manager.cc:1397] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:56.295017 9006 raft_consensus.cc:348] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.295167 9006 raft_consensus.cc:374] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:56.295212 9006 raft_consensus.cc:729] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be99fa38df004be29e01f1dd07c81b41, State: Initialized, Role: FOLLOWER
I20260808 17:45:56.295333 9006 consensus_queue.cc:260] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.295627 8958 heartbeater.cc:503] Master 127.29.62.254:42553 was elected leader, sending a full tablet report...
I20260808 17:45:56.296384 9006 ts_tablet_manager.cc:1428] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:45:56.296675 9011 raft_consensus.cc:348] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.296868 9011 raft_consensus.cc:374] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:56.296916 9011 raft_consensus.cc:729] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4adc4acd0546f79732e4e9430c2484, State: Initialized, Role: FOLLOWER
I20260808 17:45:56.297053 9011 consensus_queue.cc:260] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.297428 9011 ts_tablet_manager.cc:1428] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:45:56.297367 9009 raft_consensus.cc:348] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.297480 8724 heartbeater.cc:503] Master 127.29.62.254:42553 was elected leader, sending a full tablet report...
I20260808 17:45:56.297519 9009 raft_consensus.cc:374] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:56.297554 9009 raft_consensus.cc:729] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c78ccb5ccf415c96be50fd36c64f17, State: Initialized, Role: FOLLOWER
I20260808 17:45:56.297655 9009 consensus_queue.cc:260] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.297943 9009 ts_tablet_manager.cc:1428] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 17:45:56.298041 8496 heartbeater.cc:503] Master 127.29.62.254:42553 was elected leader, sending a full tablet report...
W20260808 17:45:56.375286 8729 tablet.cc:2369] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:45:56.458528 8498 tablet.cc:2369] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:45:56.504710 8961 tablet.cc:2369] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:45:56.564983 9018 raft_consensus.cc:482] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:56.565106 9018 raft_consensus.cc:504] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.565578 9018 leader_election.cc:288] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579), b9c78ccb5ccf415c96be50fd36c64f17 (127.29.62.193:42193)
I20260808 17:45:56.569547 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.565915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:48925 (local address 127.29.62.193:42193)
0808 17:45:56.565990 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.565992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.566002 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:45:56.566046 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.566047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.566056 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.566114 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:56.566287 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.567057 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.568591 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.569139 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.569217 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.569226 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.569432 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.569475 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 17:45:56.569547 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.565817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:38091 (local address 127.29.62.194:44579)
0808 17:45:56.565864 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.565866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.565875 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:56.565902 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.565903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.565912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.565985 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:56.566200 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.567042 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.568591 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.569139 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.569217 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.569226 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.569434 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.569476 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:45:56.569550 8964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.565777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42193 (local address 127.29.62.195:48925)
0808 17:45:56.565820 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.565834 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.565879 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.566157 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.566158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.566160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:56.566209 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.566210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.567161 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.567163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.568478 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.568482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.569241 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.569242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.569353 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.569356 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.569410 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.569447 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":11}
I20260808 17:45:56.569551 8963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.565725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:44579 (local address 127.29.62.195:38091)
0808 17:45:56.565767 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:56.565778 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:56.565828 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:56.566005 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:56.566006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:56.566007 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:56.566095 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.566127 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.567161 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.567163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.568478 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:56.568482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.569241 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.569242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:56.569329 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.569334 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:45:56.569402 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 17:45:56.569447 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":73}
I20260808 17:45:56.569972 8418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c5edf24ac84541bf72358472083459" candidate_uuid: "be99fa38df004be29e01f1dd07c81b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" is_pre_election: true
I20260808 17:45:56.570139 8418 raft_consensus.cc:2454] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be99fa38df004be29e01f1dd07c81b41 in term 0.
I20260808 17:45:56.570200 8647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c5edf24ac84541bf72358472083459" candidate_uuid: "be99fa38df004be29e01f1dd07c81b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb4adc4acd0546f79732e4e9430c2484" is_pre_election: true
I20260808 17:45:56.570406 8647 raft_consensus.cc:2454] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be99fa38df004be29e01f1dd07c81b41 in term 0.
I20260808 17:45:56.570417 8760 leader_election.cc:302] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9c78ccb5ccf415c96be50fd36c64f17, be99fa38df004be29e01f1dd07c81b41; no voters:
I20260808 17:45:56.570540 9018 raft_consensus.cc:2781] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:56.570628 9018 raft_consensus.cc:482] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:56.570667 9018 raft_consensus.cc:3037] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:56.571370 9018 raft_consensus.cc:504] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.571523 9018 leader_election.cc:288] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [CANDIDATE]: Term 1 election: Requested vote from peers cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579), b9c78ccb5ccf415c96be50fd36c64f17 (127.29.62.193:42193)
I20260808 17:45:56.571717 8418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c5edf24ac84541bf72358472083459" candidate_uuid: "be99fa38df004be29e01f1dd07c81b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c78ccb5ccf415c96be50fd36c64f17"
I20260808 17:45:56.571807 8418 raft_consensus.cc:3037] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:56.571767 8647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c5edf24ac84541bf72358472083459" candidate_uuid: "be99fa38df004be29e01f1dd07c81b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb4adc4acd0546f79732e4e9430c2484"
I20260808 17:45:56.571867 8647 raft_consensus.cc:3037] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:56.572597 8418 raft_consensus.cc:2454] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be99fa38df004be29e01f1dd07c81b41 in term 1.
I20260808 17:45:56.572600 8647 raft_consensus.cc:2454] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be99fa38df004be29e01f1dd07c81b41 in term 1.
I20260808 17:45:56.572834 8760 leader_election.cc:302] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9c78ccb5ccf415c96be50fd36c64f17, be99fa38df004be29e01f1dd07c81b41; no voters:
I20260808 17:45:56.572995 9018 raft_consensus.cc:2781] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:56.573166 9018 raft_consensus.cc:686] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 1 LEADER]: Becoming Leader. State: Replica: be99fa38df004be29e01f1dd07c81b41, State: Running, Role: LEADER
I20260808 17:45:56.573272 9018 consensus_queue.cc:237] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } }
I20260808 17:45:56.574152 6927 catalog_manager.cc:5495] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 reported cstate change: term changed from 0 to 1, leader changed from <none> to be99fa38df004be29e01f1dd07c81b41 (127.29.62.195). New cstate: current_term: 1 leader_uuid: "be99fa38df004be29e01f1dd07c81b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } health_report { overall_health: HEALTHY } } }
I20260808 17:45:56.635536 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.633072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.29.62.195:45125)
0808 17:45:56.633140 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.633142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.633151 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:56.633197 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.633198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.633216 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.633291 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:56.633449 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.633938 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.634696 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.634894 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.634984 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.635154 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:56.635261 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:56.635360 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.635429 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.635450 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:45:56.637444 8647 raft_consensus.cc:1261] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 1 FOLLOWER]: Refusing update from remote peer be99fa38df004be29e01f1dd07c81b41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:56.637490 8418 raft_consensus.cc:1261] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 FOLLOWER]: Refusing update from remote peer be99fa38df004be29e01f1dd07c81b41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:56.637840 9021 consensus_queue.cc:1035] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:56.637989 9018 consensus_queue.cc:1035] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:56.642050 9030 mvcc.cc:205] Tried to move back new op lower bound from 7316320897584009216 to 7316320897324687360. Current Snapshot: MvccSnapshot[applied={T|T < 7316320897584009216}]
I20260808 17:45:56.643111 6927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:56.643208 6927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:56.644475 6927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef37bce3589b40349d7954d1bb59e228: Sending DeleteTablet for 3 replicas of tablet 14c5edf24ac84541bf72358472083459
I20260808 17:45:56.644768 8391 tablet_service.cc:1514] Processing DeleteTablet for tablet 14c5edf24ac84541bf72358472083459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:56 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:40994
I20260808 17:45:56.644788 8609 tablet_service.cc:1514] Processing DeleteTablet for tablet 14c5edf24ac84541bf72358472083459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:56 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:33074
I20260808 17:45:56.645262 8835 tablet_service.cc:1514] Processing DeleteTablet for tablet 14c5edf24ac84541bf72358472083459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:56 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:55734
I20260808 17:45:56.645509 9043 tablet_replica.cc:330] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: stopping tablet replica
I20260808 17:45:56.645633 9041 tablet_replica.cc:330] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: stopping tablet replica
I20260808 17:45:56.645650 9043 raft_consensus.cc:2229] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:56.645743 9041 raft_consensus.cc:2229] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:56.645856 9041 raft_consensus.cc:2258] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:56.645879 9043 raft_consensus.cc:2258] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:56.646334 9041 ts_tablet_manager.cc:1905] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:56.646612 9043 ts_tablet_manager.cc:1905] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:56.646663 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 6875
I20260808 17:45:56.648497 9043 ts_tablet_manager.cc:1918] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:56.648581 9041 ts_tablet_manager.cc:1918] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:56.648681 9041 log.cc:1196] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/14c5edf24ac84541bf72358472083459
I20260808 17:45:56.648694 9043 log.cc:1196] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/14c5edf24ac84541bf72358472083459
I20260808 17:45:56.649061 9041 ts_tablet_manager.cc:1939] T 14c5edf24ac84541bf72358472083459 P b9c78ccb5ccf415c96be50fd36c64f17: Deleting consensus metadata
I20260808 17:45:56.649169 9043 ts_tablet_manager.cc:1939] T 14c5edf24ac84541bf72358472083459 P be99fa38df004be29e01f1dd07c81b41: Deleting consensus metadata
I20260808 17:45:56.649520 9042 tablet_replica.cc:330] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: stopping tablet replica
I20260808 17:45:56.649703 9042 raft_consensus.cc:2229] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:56.649835 9042 raft_consensus.cc:2258] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:56.650470 9042 ts_tablet_manager.cc:1905] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:56.652143 9042 ts_tablet_manager.cc:1918] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:56.652204 9042 log.cc:1196] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/14c5edf24ac84541bf72358472083459
I20260808 17:45:56.652431 9042 ts_tablet_manager.cc:1939] T 14c5edf24ac84541bf72358472083459 P cb4adc4acd0546f79732e4e9430c2484: Deleting consensus metadata
I20260808 17:45:56.661401 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.646469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42026 (local address 127.29.62.252:46541)
0808 17:45:56.646512 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.646514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.654684 (+ 8170us) server_negotiation.cc:407] Connection header received
0808 17:45:56.654768 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.654769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.654785 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.654843 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.655509 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.656096 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.656803 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.657080 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.657160 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.658727 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.658729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.658730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.658738 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.660222 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.660594 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.660595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.660596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.660658 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.660867 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.660869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.660869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.660991 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.661099 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.661208 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.661296 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:45:56.661499 8761 connection.cc:535] server connection from 127.0.0.1:55734 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 17:45:56.661732 8309 connection.cc:535] server connection from 127.0.0.1:40994 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 17:45:56.661942 8539 connection.cc:535] server connection from 127.0.0.1:33074 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:45:56.670344 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.663766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.29.62.252:46541)
0808 17:45:56.663870 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.663872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.663879 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:45:56.663898 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.663898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.663914 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.663966 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.664533 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.665101 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.665890 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.666094 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.666168 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.667562 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.667563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.667564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.667570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.669034 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.669429 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.669430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.669431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.669490 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.669716 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.669717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.669718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.669857 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.669932 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.670123 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.670201 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 17:45:56.671589 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:56.663821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.29.62.253:44951)
0808 17:45:56.663897 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:56.663899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:56.663908 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:56.663935 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:56.663936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:56.663950 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:56.663999 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:56.665087 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.665875 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.666989 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:56.667366 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:56.667445 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:56.668783 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:56.668785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:56.668785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:56.668793 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:56.670498 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.670866 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.670866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.670867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.670930 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:56.671158 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:56.671158 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:56.671159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:56.671261 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:56.671330 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:56.671462 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 17:45:56.671529 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 17:45:57.642042 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:57.641730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.194:40211)
0808 17:45:57.641882 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:57.641979 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
W20260808 17:45:57.642318 8724 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:42553 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
I20260808 17:45:57.644101 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:57.643793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.193:57715)
0808 17:45:57.643940 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:57.644045 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":54,"threads_started":1}
I20260808 17:45:57.644191 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:57.643909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.195:45495)
0808 17:45:57.644053 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:57.644142 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
W20260808 17:45:57.644377 8496 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:42553 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
W20260808 17:45:57.644522 8958 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:42553 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
I20260808 17:45:58.145728 9059 raft_consensus.cc:482] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef37bce3589b40349d7954d1bb59e228)
I20260808 17:45:58.145916 9059 raft_consensus.cc:504] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } }
I20260808 17:45:58.146015 9063 raft_consensus.cc:482] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef37bce3589b40349d7954d1bb59e228)
I20260808 17:45:58.146189 9063 raft_consensus.cc:504] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } }
I20260808 17:45:58.146298 9059 leader_election.cc:288] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef37bce3589b40349d7954d1bb59e228 (127.29.62.254:42553), bac033c9a915486d8910ac4bbc5636dd (127.29.62.253:44951)
I20260808 17:45:58.146528 9063 leader_election.cc:288] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef37bce3589b40349d7954d1bb59e228 (127.29.62.254:42553), f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252:46541)
I20260808 17:45:58.146999 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.146628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:46070)
0808 17:45:58.146881 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.146951 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":54,"threads_started":1}
W20260808 17:45:58.147117 7875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
I20260808 17:45:58.147265 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.146988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:46058)
0808 17:45:58.147175 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.147222 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":43,"threads_started":1}
W20260808 17:45:58.147456 8182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
I20260808 17:45:58.147590 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.147483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:46084)
0808 17:45:58.147534 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.147562 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 17:45:58.147658 7875 leader_election.cc:334] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef37bce3589b40349d7954d1bb59e228 (127.29.62.254:42553): Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
I20260808 17:45:58.148094 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.147981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:46100)
0808 17:45:58.148035 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.148063 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 17:45:58.148178 8182 leader_election.cc:334] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef37bce3589b40349d7954d1bb59e228 (127.29.62.254:42553): Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
I20260808 17:45:58.150122 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.146677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:46541 (local address 127.0.0.1:42050)
0808 17:45:58.147374 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.147384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:58.147470 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:58.147648 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:58.147649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:58.147651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:58.147712 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.147716 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.148576 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.148577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.149317 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.149319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.149865 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.149866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.149932 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.149937 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:45:58.150011 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:45:58.150056 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":48,"threads_started":1}
I20260808 17:45:58.150130 9069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.146873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.29.62.252:46541)
0808 17:45:58.147046 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:58.147048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:58.147437 (+ 389us) server_negotiation.cc:407] Connection header received
0808 17:45:58.147542 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:58.147544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:58.147555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:58.147645 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:58.147823 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.148491 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.149415 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.149782 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.149863 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.149875 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:58.150017 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 17:45:58.150058 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":56,"threads_started":1}
I20260808 17:45:58.150398 9067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.146731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.29.62.253:44951)
0808 17:45:58.147229 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:58.147232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:58.147241 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:58.147267 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:58.147267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:58.147280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:58.148058 (+ 778us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:58.148096 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.148704 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.149641 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.150001 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.150078 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.150088 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:58.150231 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 17:45:58.150291 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":37,"threads_started":1}
I20260808 17:45:58.150398 8218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bac033c9a915486d8910ac4bbc5636dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f750d1c0045a4c9a93ed3cb852accb33" is_pre_election: true
I20260808 17:45:58.150405 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.146518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:44951 (local address 127.0.0.1:51838)
0808 17:45:58.146631 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.146644 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:58.146709 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:58.147829 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:58.147830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:58.147833 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:58.147890 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.147892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.148795 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.148797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.149540 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.149542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.150080 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.150081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.150149 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.150152 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:58.150223 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:45:58.150290 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":49,"threads_started":1}
I20260808 17:45:58.150533 8218 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bac033c9a915486d8910ac4bbc5636dd in term 1.
I20260808 17:45:58.150658 7874 leader_election.cc:302] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bac033c9a915486d8910ac4bbc5636dd, f750d1c0045a4c9a93ed3cb852accb33; no voters: ef37bce3589b40349d7954d1bb59e228
I20260808 17:45:58.150772 9063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 17:45:58.150837 9063 raft_consensus.cc:482] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef37bce3589b40349d7954d1bb59e228)
I20260808 17:45:58.150776 7906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f750d1c0045a4c9a93ed3cb852accb33" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bac033c9a915486d8910ac4bbc5636dd" is_pre_election: true
I20260808 17:45:58.150861 9063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:45:58.151674 9063 raft_consensus.cc:504] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } }
I20260808 17:45:58.151815 7906 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f750d1c0045a4c9a93ed3cb852accb33 in current term 2: Already voted for candidate bac033c9a915486d8910ac4bbc5636dd in this term.
I20260808 17:45:58.151829 9063 leader_election.cc:288] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [CANDIDATE]: Term 2 election: Requested vote from peers ef37bce3589b40349d7954d1bb59e228 (127.29.62.254:42553), f750d1c0045a4c9a93ed3cb852accb33 (127.29.62.252:46541)
I20260808 17:45:58.151994 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.151899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:46108)
0808 17:45:58.151938 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.151970 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 17:45:58.151969 8218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bac033c9a915486d8910ac4bbc5636dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f750d1c0045a4c9a93ed3cb852accb33"
I20260808 17:45:58.152040 8218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:45:58.152310 8179 leader_election.cc:302] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f750d1c0045a4c9a93ed3cb852accb33; no voters: bac033c9a915486d8910ac4bbc5636dd, ef37bce3589b40349d7954d1bb59e228
I20260808 17:45:58.152392 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.152288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:46124)
0808 17:45:58.152339 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.152365 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 17:45:58.152469 7875 leader_election.cc:334] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef37bce3589b40349d7954d1bb59e228 (127.29.62.254:42553): Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
I20260808 17:45:58.152812 8218 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bac033c9a915486d8910ac4bbc5636dd in term 2.
I20260808 17:45:58.152892 9059 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:45:58.153005 7874 leader_election.cc:302] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bac033c9a915486d8910ac4bbc5636dd, f750d1c0045a4c9a93ed3cb852accb33; no voters: ef37bce3589b40349d7954d1bb59e228
I20260808 17:45:58.153189 9063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:45:58.153332 9063 raft_consensus.cc:686] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [term 2 LEADER]: Becoming Leader. State: Replica: bac033c9a915486d8910ac4bbc5636dd, State: Running, Role: LEADER
I20260808 17:45:58.153421 9063 consensus_queue.cc:237] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } }
I20260808 17:45:58.153759 9073 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader bac033c9a915486d8910ac4bbc5636dd. Latest consensus state: current_term: 2 leader_uuid: "bac033c9a915486d8910ac4bbc5636dd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:58.153836 9073 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:58.153999 9076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:58.154503 9076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:58.154595 9076 catalog_manager.cc:1259] Loaded cluster ID: 11c55d0c6a7040a39622e80b0dfcf358
I20260808 17:45:58.154628 9076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:45:58.154784 9076 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:58.154879 9076 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Loaded TSK: 0
I20260808 17:45:58.155046 9076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:45:58.208549 7891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:45:58.210353 7891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:45:58.211483 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.211355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:46134)
0808 17:45:58.211406 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.211435 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:45:58.211535 8218 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [term 2 FOLLOWER]: Refusing update from remote peer bac033c9a915486d8910ac4bbc5636dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 17:45:58.211853 9073 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 17:45:58.211949 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.211845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.0.0.1:46138)
0808 17:45:58.211892 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.211918 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 17:45:58.212036 7875 consensus_peers.cc:487] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd -> Peer ef37bce3589b40349d7954d1bb59e228 (127.29.62.254:42553): Couldn't send request to peer ef37bce3589b40349d7954d1bb59e228. Status: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 17:45:58.212653 9059 sys_catalog.cc:455] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bac033c9a915486d8910ac4bbc5636dd. Latest consensus state: current_term: 2 leader_uuid: "bac033c9a915486d8910ac4bbc5636dd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:58.212769 9059 sys_catalog.cc:458] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:45:58.213238 9073 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bac033c9a915486d8910ac4bbc5636dd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:58.213336 9073 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:58.213307 9059 sys_catalog.cc:455] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bac033c9a915486d8910ac4bbc5636dd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ef37bce3589b40349d7954d1bb59e228" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42553 } } peers { permanent_uuid: "bac033c9a915486d8910ac4bbc5636dd" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 44951 } attrs { promote: false } } peers { permanent_uuid: "f750d1c0045a4c9a93ed3cb852accb33" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 46541 } attrs { promote: false } } }
I20260808 17:45:58.213395 9059 sys_catalog.cc:458] T 00000000000000000000000000000000 P f750d1c0045a4c9a93ed3cb852accb33 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:45:58.213553 9082 mvcc.cc:205] Tried to move back new op lower bound from 7316320904032215040 to 7316320903797022720. Current Snapshot: MvccSnapshot[applied={T|T < 7316320904032215040}]
I20260808 17:45:58.218345 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.215214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45125 (local address 127.0.0.1:55748)
0808 17:45:58.215312 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.215322 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:58.215364 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:58.215567 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:58.215568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:58.215569 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:58.215625 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.215627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.216431 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.216432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.217381 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.217384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.218130 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.218131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.218189 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.218193 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:58.218245 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:45:58.218281 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 17:45:58.218386 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.215317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.29.62.195:45125)
0808 17:45:58.215443 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:58.215445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:58.215452 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:45:58.215481 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:58.215482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:58.215495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:58.215560 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:58.215726 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.216346 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.217476 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.218059 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.218134 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.218147 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:58.218252 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 17:45:58.218291 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260808 17:45:58.218600 8835 tablet_service.cc:1467] Processing CreateTablet for tablet bde7fb4624a54afbad31876a0a800866 (DEFAULT_TABLE table=test-table [id=c3c4c9adc09e4256ab6f2e676fcb5ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:58.218736 8835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bde7fb4624a54afbad31876a0a800866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:58.218835 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.215283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42193 (local address 127.0.0.1:41002)
0808 17:45:58.215385 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.215395 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:58.215444 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:58.215830 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:58.215831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:58.215834 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:58.215885 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.215887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.216850 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.216850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.217801 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.217804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.218625 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.218627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.218689 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.218693 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:58.218722 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 17:45:58.218759 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":37}
I20260808 17:45:58.219152 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.215527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41002 (local address 127.29.62.193:42193)
0808 17:45:58.215684 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:58.215686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:58.215695 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:45:58.215723 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:58.215724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:58.215736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:58.215965 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:58.216022 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.216746 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.217888 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.218527 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.219018 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.219033 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:58.219054 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 17:45:58.219102 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":57,"threads_started":1}
I20260808 17:45:58.219157 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.215498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33080 (local address 127.29.62.194:44579)
0808 17:45:58.215621 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:58.215623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:58.215696 (+ 73us) server_negotiation.cc:407] Connection header received
0808 17:45:58.215799 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:58.215800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:58.215812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:58.215850 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:58.216051 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.216765 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.218059 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.218746 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.218812 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.218824 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:58.219040 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 17:45:58.219082 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
I20260808 17:45:58.219357 8391 tablet_service.cc:1467] Processing CreateTablet for tablet bde7fb4624a54afbad31876a0a800866 (DEFAULT_TABLE table=test-table [id=c3c4c9adc09e4256ab6f2e676fcb5ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:58.219408 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.215306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:44579 (local address 127.0.0.1:33080)
0808 17:45:58.215639 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.215649 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:58.215723 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:58.215911 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:58.215912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:58.215914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:58.215958 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.215959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.216846 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.216847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.217969 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.217972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.218860 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.218861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.218949 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.218954 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:45:58.219276 (+ 322us) client_negotiation.cc:240] Negotiation successful
0808 17:45:58.219322 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":23,"thread_start_us":84,"threads_started":1}
I20260808 17:45:58.219475 8391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bde7fb4624a54afbad31876a0a800866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:58.219650 8609 tablet_service.cc:1467] Processing CreateTablet for tablet bde7fb4624a54afbad31876a0a800866 (DEFAULT_TABLE table=test-table [id=c3c4c9adc09e4256ab6f2e676fcb5ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:45:58.219779 8609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bde7fb4624a54afbad31876a0a800866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:58.220341 9097 tablet_bootstrap.cc:492] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: Bootstrap starting.
I20260808 17:45:58.220824 9097 tablet_bootstrap.cc:654] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:58.221005 9098 tablet_bootstrap.cc:492] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17: Bootstrap starting.
I20260808 17:45:58.221367 9098 tablet_bootstrap.cc:654] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:58.221587 9097 tablet_bootstrap.cc:492] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: No bootstrap required, opened a new log
I20260808 17:45:58.221655 9097 ts_tablet_manager.cc:1397] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:58.221762 9099 tablet_bootstrap.cc:492] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: Bootstrap starting.
I20260808 17:45:58.221774 9097 raft_consensus.cc:348] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.221868 9097 raft_consensus.cc:374] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:58.221889 9097 raft_consensus.cc:729] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be99fa38df004be29e01f1dd07c81b41, State: Initialized, Role: FOLLOWER
I20260808 17:45:58.221938 9097 consensus_queue.cc:260] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.222096 9097 ts_tablet_manager.cc:1428] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:58.222133 9099 tablet_bootstrap.cc:654] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:58.222728 8960 heartbeater.cc:503] Master 127.29.62.253:44951 was elected leader, sending a full tablet report...
I20260808 17:45:58.223110 9098 tablet_bootstrap.cc:492] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17: No bootstrap required, opened a new log
I20260808 17:45:58.223217 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.222270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.195:47813)
0808 17:45:58.222654 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.222693 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":80,"threads_started":1}
I20260808 17:45:58.223215 9098 ts_tablet_manager.cc:1397] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:58.223304 9099 tablet_bootstrap.cc:492] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: No bootstrap required, opened a new log
I20260808 17:45:58.223363 9099 ts_tablet_manager.cc:1397] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:45:58.223497 9098 raft_consensus.cc:348] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.223518 9099 raft_consensus.cc:348] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.223557 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.223470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.195:35179)
0808 17:45:58.223505 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.223537 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 17:45:58.223589 9098 raft_consensus.cc:374] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:58.223608 9099 raft_consensus.cc:374] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:58.223613 9098 raft_consensus.cc:729] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c78ccb5ccf415c96be50fd36c64f17, State: Initialized, Role: FOLLOWER
I20260808 17:45:58.223634 9099 raft_consensus.cc:729] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4adc4acd0546f79732e4e9430c2484, State: Initialized, Role: FOLLOWER
I20260808 17:45:58.223662 9098 consensus_queue.cc:260] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.223694 9099 consensus_queue.cc:260] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.223848 9098 ts_tablet_manager.cc:1428] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 17:45:58.224285 9099 ts_tablet_manager.cc:1428] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:45:58.224387 8497 heartbeater.cc:503] Master 127.29.62.253:44951 was elected leader, sending a full tablet report...
I20260808 17:45:58.224813 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.223964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.193:48237)
0808 17:45:58.224704 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.224744 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":86,"threads_started":1}
I20260808 17:45:58.224988 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.224486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.194:40675)
0808 17:45:58.224896 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.224944 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":82,"threads_started":1}
I20260808 17:45:58.225208 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.225067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.193:49101)
0808 17:45:58.225149 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.225181 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 17:45:58.225339 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.225261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.194:55237)
0808 17:45:58.225297 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.225319 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 17:45:58.225467 8726 heartbeater.cc:503] Master 127.29.62.253:44951 was elected leader, sending a full tablet report...
I20260808 17:45:58.239689 9106 raft_consensus.cc:482] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:45:58.239764 9106 raft_consensus.cc:504] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.240090 9106 leader_election.cc:288] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be99fa38df004be29e01f1dd07c81b41 (127.29.62.195:45125), cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579)
I20260808 17:45:58.243938 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.240360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:37845 (local address 127.29.62.194:44579)
0808 17:45:58.240407 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:58.240409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:58.240526 (+ 117us) server_negotiation.cc:407] Connection header received
0808 17:45:58.240569 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:58.240570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:58.240578 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:58.240632 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:58.240886 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.241643 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.243171 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.243573 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.243664 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.243675 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:58.243832 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 17:45:58.243872 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:45:58.244038 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.240253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:45833 (local address 127.29.62.195:45125)
0808 17:45:58.240320 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:58.240322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:58.240442 (+ 120us) server_negotiation.cc:407] Connection header received
0808 17:45:58.240535 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:58.240536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:58.240547 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:58.240612 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:45:58.240886 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.241654 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.243238 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.243639 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.243731 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.243744 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:58.243922 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 17:45:58.243966 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:45:58.243973 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.240252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:44579 (local address 127.29.62.193:37845)
0808 17:45:58.240443 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.240452 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:58.240517 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:58.240648 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:58.240649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:58.240651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:58.240787 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.240789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.241739 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.241740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.243096 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.243098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.243664 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.243665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.243750 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.243754 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:45:58.243822 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 17:45:58.243866 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":78,"thread_start_us":113,"threads_started":1}
I20260808 17:45:58.244338 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.240208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45125 (local address 127.29.62.193:45833)
0808 17:45:58.240341 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.240351 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:45:58.240419 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:45:58.240648 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:45:58.240649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:45:58.240654 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:45:58.240787 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.240789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.241739 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.241740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.243087 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:45:58.243092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.243755 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.243756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:45:58.243838 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.243841 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:45:58.243911 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 17:45:58.243955 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":65}
I20260808 17:45:58.244529 8647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bde7fb4624a54afbad31876a0a800866" candidate_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb4adc4acd0546f79732e4e9430c2484" is_pre_election: true
I20260808 17:45:58.244644 8647 raft_consensus.cc:2454] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9c78ccb5ccf415c96be50fd36c64f17 in term 0.
I20260808 17:45:58.244565 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bde7fb4624a54afbad31876a0a800866" candidate_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be99fa38df004be29e01f1dd07c81b41" is_pre_election: true
I20260808 17:45:58.244716 8869 raft_consensus.cc:2454] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9c78ccb5ccf415c96be50fd36c64f17 in term 0.
I20260808 17:45:58.244849 8311 leader_election.cc:302] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9c78ccb5ccf415c96be50fd36c64f17, cb4adc4acd0546f79732e4e9430c2484; no voters:
I20260808 17:45:58.245019 9106 raft_consensus.cc:2781] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:45:58.245060 9106 raft_consensus.cc:482] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:45:58.245076 9106 raft_consensus.cc:3037] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:58.245738 9106 raft_consensus.cc:504] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.245883 9106 leader_election.cc:288] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [CANDIDATE]: Term 1 election: Requested vote from peers be99fa38df004be29e01f1dd07c81b41 (127.29.62.195:45125), cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579)
I20260808 17:45:58.246025 8869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bde7fb4624a54afbad31876a0a800866" candidate_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be99fa38df004be29e01f1dd07c81b41"
I20260808 17:45:58.246047 8647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bde7fb4624a54afbad31876a0a800866" candidate_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb4adc4acd0546f79732e4e9430c2484"
I20260808 17:45:58.246104 8869 raft_consensus.cc:3037] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:58.246114 8647 raft_consensus.cc:3037] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:58.246925 8647 raft_consensus.cc:2454] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9c78ccb5ccf415c96be50fd36c64f17 in term 1.
I20260808 17:45:58.246973 8869 raft_consensus.cc:2454] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9c78ccb5ccf415c96be50fd36c64f17 in term 1.
I20260808 17:45:58.247107 8309 leader_election.cc:302] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9c78ccb5ccf415c96be50fd36c64f17, be99fa38df004be29e01f1dd07c81b41; no voters:
I20260808 17:45:58.247215 9106 raft_consensus.cc:2781] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:58.247393 9106 raft_consensus.cc:686] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 LEADER]: Becoming Leader. State: Replica: b9c78ccb5ccf415c96be50fd36c64f17, State: Running, Role: LEADER
I20260808 17:45:58.247478 9106 consensus_queue.cc:237] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } }
I20260808 17:45:58.247956 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.247833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.193:35281)
0808 17:45:58.247891 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.247923 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 17:45:58.248270 7891 catalog_manager.cc:5495] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9c78ccb5ccf415c96be50fd36c64f17 (127.29.62.193). New cstate: current_term: 1 leader_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9c78ccb5ccf415c96be50fd36c64f17" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 42193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 } health_report { overall_health: UNKNOWN } } }
I20260808 17:45:58.255218 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.252582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.29.62.193:42193)
0808 17:45:58.252643 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:58.252645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:58.252656 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:45:58.252691 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:58.252692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:58.252713 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:58.252793 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:45:58.252927 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.253666 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.254527 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:58.254713 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:58.254768 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:58.254884 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:45:58.255002 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:45:58.255053 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:58.255092 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 17:45:58.255110 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:45:58.256227 8647 raft_consensus.cc:1261] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 1 FOLLOWER]: Refusing update from remote peer b9c78ccb5ccf415c96be50fd36c64f17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:58.256325 8869 raft_consensus.cc:1261] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 1 FOLLOWER]: Refusing update from remote peer b9c78ccb5ccf415c96be50fd36c64f17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:45:58.256523 9117 consensus_queue.cc:1035] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb4adc4acd0546f79732e4e9430c2484" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 44579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:58.256645 9106 consensus_queue.cc:1035] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be99fa38df004be29e01f1dd07c81b41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:45:58.257397 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.257288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.195:46255)
0808 17:45:58.257325 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.257362 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 17:45:58.257939 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.257456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.194:41723)
0808 17:45:58.257859 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.257895 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384}
I20260808 17:45:58.258920 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.258792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.194:58449)
0808 17:45:58.258853 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.258881 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 17:45:58.259277 9124 mvcc.cc:205] Tried to move back new op lower bound from 7316320904215560192 to 7316320904182415360. Current Snapshot: MvccSnapshot[applied={T|T < 7316320904215560192}]
I20260808 17:45:58.259536 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.259448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.195:35685)
0808 17:45:58.259488 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.259511 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 17:45:58.260792 7889 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:58.260885 7889 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:45:58.261268 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.261173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42553 (local address 127.29.62.193:50153)
0808 17:45:58.261206 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.261236 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:42553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 17:45:58.262262 7889 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bac033c9a915486d8910ac4bbc5636dd: Sending DeleteTablet for 3 replicas of tablet bde7fb4624a54afbad31876a0a800866
I20260808 17:45:58.262547 8391 tablet_service.cc:1514] Processing DeleteTablet for tablet bde7fb4624a54afbad31876a0a800866 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:58 UTC) from {username='oryx', principal='oryx/127.29.62.253@KRBTEST.COM'} at 127.0.0.1:41002
I20260808 17:45:58.262611 8609 tablet_service.cc:1514] Processing DeleteTablet for tablet bde7fb4624a54afbad31876a0a800866 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:58 UTC) from {username='oryx', principal='oryx/127.29.62.253@KRBTEST.COM'} at 127.0.0.1:33080
I20260808 17:45:58.262782 9150 tablet_replica.cc:330] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17: stopping tablet replica
I20260808 17:45:58.262840 9150 raft_consensus.cc:2229] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:45:58.262926 9150 raft_consensus.cc:2258] T bde7fb4624a54afbad31876a0a800866 P b9c78ccb5ccf415c96be50fd36c64f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:58.262957 8835 tablet_service.cc:1514] Processing DeleteTablet for tablet bde7fb4624a54afbad31876a0a800866 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:45:58 UTC) from {username='oryx', principal='oryx/127.29.62.253@KRBTEST.COM'} at 127.0.0.1:55748
I20260808 17:45:58.263163 9155 tablet_replica.cc:330] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: stopping tablet replica
I20260808 17:45:58.263223 9155 raft_consensus.cc:2229] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:58.263273 9155 raft_consensus.cc:2258] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:58.263583 9155 ts_tablet_manager.cc:1905] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:58.263703 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 8132
I20260808 17:45:58.272991 9151 tablet_replica.cc:330] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: stopping tablet replica
I20260808 17:45:58.273078 9151 raft_consensus.cc:2229] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:45:58.273144 9151 raft_consensus.cc:2258] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:45:58.273468 9155 ts_tablet_manager.cc:1918] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:58.273545 9155 log.cc:1196] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/bde7fb4624a54afbad31876a0a800866
I20260808 17:45:58.273535 9151 ts_tablet_manager.cc:1905] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:45:58.273801 9155 ts_tablet_manager.cc:1939] T bde7fb4624a54afbad31876a0a800866 P be99fa38df004be29e01f1dd07c81b41: Deleting consensus metadata
I20260808 17:45:58.274257 7873 catalog_manager.cc:4841] TS be99fa38df004be29e01f1dd07c81b41 (127.29.62.195:45125): tablet bde7fb4624a54afbad31876a0a800866 (table test-table [id=c3c4c9adc09e4256ab6f2e676fcb5ea7]) successfully deleted
I20260808 17:45:58.275009 9151 ts_tablet_manager.cc:1918] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:45:58.275065 9151 log.cc:1196] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/bde7fb4624a54afbad31876a0a800866
I20260808 17:45:58.275306 9151 ts_tablet_manager.cc:1939] T bde7fb4624a54afbad31876a0a800866 P cb4adc4acd0546f79732e4e9430c2484: Deleting consensus metadata
I20260808 17:45:58.275673 7875 catalog_manager.cc:4841] TS cb4adc4acd0546f79732e4e9430c2484 (127.29.62.194:44579): tablet bde7fb4624a54afbad31876a0a800866 (table test-table [id=c3c4c9adc09e4256ab6f2e676fcb5ea7]) successfully deleted
I20260808 17:45:58.276800 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:58.276652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:42193 (local address 127.0.0.1:41024)
0808 17:45:58.276721 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 17:45:58.276756 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 17:45:58.276948 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 8505
W20260808 17:45:58.276979 7874 catalog_manager.cc:4584] TS b9c78ccb5ccf415c96be50fd36c64f17 (127.29.62.193:42193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bde7fb4624a54afbad31876a0a800866: Network error: Client connection negotiation failed: client connection to 127.29.62.193:42193: connect: Connection refused (error 111)
I20260808 17:45:58.287375 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 8735
I20260808 17:45:58.302052 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 7688
I20260808 17:45:58.309748 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 7975
2026-08-08T17:45:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5096 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:45:58 dist-test-slave-tk9z krb5kdc[9158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:45:58 dist-test-slave-tk9z krb5kdc[9158](info): set up 2 sockets
Aug 08 17:45:58 dist-test-slave-tk9z krb5kdc[9158](info): commencing operation
krb5kdc: starting...
W20260808 17:45:59.655026 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:45:59 dist-test-slave-tk9z krb5kdc[9158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:45:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:45:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.254.keytab.
Entry for principal oryx/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.254.keytab.
I20260808 17:45:59.777947 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:43633
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.254.keytab
--principal=oryx/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:43633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:45:59.861677 9181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:45:59.861891 9181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:45:59.861915 9181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:45:59.863162 9181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:45:59.863207 9181 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 17:45:59.863220 9181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:45:59.863235 9181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:45:59.863247 9181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:45:59.863299 9181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:45:59.864964 9181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:43633
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.254.keytab
--principal=oryx/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:43633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:45:59.865226 9181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:45:59.865466 9181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:45:59.870334 9188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:59.870326 9193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:45:59.870410 9187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:45:59.870726 9181 server_base.cc:1034] running on GCE node
I20260808 17:45:59.871135 9181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:45:59.871502 9181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:45:59.872694 9181 hybrid_clock.cc:648] HybridClock initialized: now 1786211159872668 us; error 37 us; skew 500 ppm
Aug 08 17:45:59 dist-test-slave-tk9z krb5kdc[9158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211159, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:45:59.875486 9181 init.cc:377] Logged in from keytab as oryx/127.29.62.254@KRBTEST.COM (short username oryx)
I20260808 17:45:59.875996 9181 webserver.cc:460] Webserver started at http://127.29.62.254:39389/ using document root <none> and password file <none>
I20260808 17:45:59.876231 9181 fs_manager.cc:359] Metadata directory not provided
I20260808 17:45:59.876300 9181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:45:59.876463 9181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:45:59.877559 9181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "79d48cf38cc54799aca347f97d461918"
format_stamp: "Formatted at 2026-08-08 17:45:59 on dist-test-slave-tk9z"
server_key: "4934665e5662ccebfb22717d31c44af7"
server_key_iv: "f512320b0a820ac72809fbdaa8dd634a"
server_key_version: "encryptionkey@0"
I20260808 17:45:59.877947 9181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "79d48cf38cc54799aca347f97d461918"
format_stamp: "Formatted at 2026-08-08 17:45:59 on dist-test-slave-tk9z"
server_key: "4934665e5662ccebfb22717d31c44af7"
server_key_iv: "f512320b0a820ac72809fbdaa8dd634a"
server_key_version: "encryptionkey@0"
I20260808 17:45:59.879405 9181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:45:59.880160 9204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:45:59.880335 9181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:45:59.880450 9181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "79d48cf38cc54799aca347f97d461918"
format_stamp: "Formatted at 2026-08-08 17:45:59 on dist-test-slave-tk9z"
server_key: "4934665e5662ccebfb22717d31c44af7"
server_key_iv: "f512320b0a820ac72809fbdaa8dd634a"
server_key_version: "encryptionkey@0"
I20260808 17:45:59.880529 9181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:45:59.907735 9181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:45:59.909703 9181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:45:59.909885 9181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:45:59.914949 9181 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:43633
I20260808 17:45:59.914983 9304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:43633 every 8 connection(s)
I20260808 17:45:59.915349 9181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:45:59.916087 9306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:45:59.918469 9306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: Bootstrap starting.
I20260808 17:45:59.919091 9306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: Neither blocks nor log segments found. Creating new log.
I20260808 17:45:59.919334 9306 log.cc:824] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: Log is configured to *not* fsync() on all Append() calls
I20260808 17:45:59.919988 9306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: No bootstrap required, opened a new log
I20260808 17:45:59.921995 9306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d48cf38cc54799aca347f97d461918" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43633 } }
I20260808 17:45:59.922137 9306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:45:59.922159 9306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d48cf38cc54799aca347f97d461918, State: Initialized, Role: FOLLOWER
I20260808 17:45:59.922261 9306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d48cf38cc54799aca347f97d461918" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43633 } }
I20260808 17:45:59.922348 9306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:45:59.922384 9306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:45:59.922469 9306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:45:59.923002 9306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d48cf38cc54799aca347f97d461918" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43633 } }
I20260808 17:45:59.923130 9306 leader_election.cc:302] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79d48cf38cc54799aca347f97d461918; no voters:
I20260808 17:45:59.923313 9306 leader_election.cc:288] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:45:59.923463 9310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:45:59.923619 9306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:45:59.923623 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 9181
I20260808 17:45:59.923731 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:45:59.923659 9310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [term 1 LEADER]: Becoming Leader. State: Replica: 79d48cf38cc54799aca347f97d461918, State: Running, Role: LEADER
I20260808 17:45:59.923843 9310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d48cf38cc54799aca347f97d461918" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43633 } }
I20260808 17:45:59.923955 29947 external_mini_cluster.cc:1428] Setting key 631e4c747c48e6c1d1085b571bee60dd
I20260808 17:45:59.924382 9312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79d48cf38cc54799aca347f97d461918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d48cf38cc54799aca347f97d461918" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43633 } } }
I20260808 17:45:59.924481 9312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:59.924600 9311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d48cf38cc54799aca347f97d461918. Latest consensus state: current_term: 1 leader_uuid: "79d48cf38cc54799aca347f97d461918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d48cf38cc54799aca347f97d461918" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 43633 } } }
I20260808 17:45:59.924796 9315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:45:59.925199 9311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918 [sys.catalog]: This master's current role is: LEADER
I20260808 17:45:59.925726 9315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:45:59.927440 9315 catalog_manager.cc:1347] Generated new cluster ID: 2671e457343b481e80ce9d69fce6dc06
I20260808 17:45:59.927500 9315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:45:59 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:45:59.939178 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:45:59.925273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46730 (local address 127.29.62.254:43633)
0808 17:45:59.925498 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 17:45:59.925508 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:45:59.925542 (+ 34us) server_negotiation.cc:407] Connection header received
0808 17:45:59.925690 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:45:59.925699 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:45:59.925763 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:45:59.925945 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:45:59.926914 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:59.931158 (+ 4244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:59.932029 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:45:59.932523 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:45:59.932654 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:45:59.935344 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:45:59.935349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:45:59.935349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:45:59.935363 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:45:59.937266 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:59.937690 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:59.937693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:59.937693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:59.937813 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:45:59.938066 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:45:59.938067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:45:59.938068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:45:59.938343 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:45:59.938582 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 17:45:59.938739 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 17:45:59.938905 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260808 17:45:59.939649 9315 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:45:59.940315 9315 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:45:59.947770 9315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: Generated new TSK 0
I20260808 17:45:59.947978 9315 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab.
Entry for principal oryx/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab.
I20260808 17:45:59.978087 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:43633
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab
--principal=oryx/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:00.062016 9343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:00.062232 9343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:00.062258 9343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:00.063436 9343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:00.063482 9343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:00.063503 9343 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:46:00.063521 9343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:46:00.065079 9343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab
--principal=oryx/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:43633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:00.065320 9343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:00.065572 9343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:00.070621 9348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:00.070621 9356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:00.070634 9349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:00.071079 9343 server_base.cc:1034] running on GCE node
I20260808 17:46:00.071280 9343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:00.071571 9343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:00.072760 9343 hybrid_clock.cc:648] HybridClock initialized: now 1786211160072729 us; error 45 us; skew 500 ppm
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211160, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:00.075523 9343 init.cc:377] Logged in from keytab as oryx/127.29.62.193@KRBTEST.COM (short username oryx)
I20260808 17:46:00.076026 9343 webserver.cc:460] Webserver started at http://127.29.62.193:37949/ using document root <none> and password file <none>
I20260808 17:46:00.076267 9343 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:00.076336 9343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:00.076484 9343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:00.077555 9343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "18b5658b628e47bfb8c7a8476c38ac60"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "ec1bfe97baa5ed609bbd1cec301e1028"
server_key_iv: "494f2eb60f6dbc060bc10274d2667821"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.077934 9343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "18b5658b628e47bfb8c7a8476c38ac60"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "ec1bfe97baa5ed609bbd1cec301e1028"
server_key_iv: "494f2eb60f6dbc060bc10274d2667821"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.079388 9343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:46:00.080137 9365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.080309 9343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:00.080421 9343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "18b5658b628e47bfb8c7a8476c38ac60"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "ec1bfe97baa5ed609bbd1cec301e1028"
server_key_iv: "494f2eb60f6dbc060bc10274d2667821"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.080507 9343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:00.098903 9343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:00.100781 9343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:00.100951 9343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:00.101478 9343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:00.101542 9343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.101593 9343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:00.101624 9343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211160, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:46:00.109287 9343 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:35969
I20260808 17:46:00.109372 9563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:35969 every 8 connection(s)
I20260808 17:46:00.109735 9343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:46:00.111724 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.102048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:46293 (local address 127.29.62.254:43633)
0808 17:46:00.102093 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.102095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.102343 (+ 248us) server_negotiation.cc:407] Connection header received
0808 17:46:00.102917 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.102919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.102936 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.103001 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.103856 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.104711 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.105792 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.106137 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.106206 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.108553 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:00.108555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:00.108555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:00.108565 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:00.110352 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.110774 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.110774 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.110775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.110827 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.111032 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.111033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.111033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.111192 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:00.111286 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.111576 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.111657 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:46:00.111905 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.102047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43633 (local address 127.29.62.193:46293)
0808 17:46:00.102218 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.102253 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.102805 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.103081 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.103084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.103433 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:00.103728 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.103734 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.104830 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.104831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.105691 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.105694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.106239 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.106241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.106417 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.106954 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:00.106964 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:00.108455 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:00.110486 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.110489 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.110491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.110706 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.110888 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.110888 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.110888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.110976 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.111258 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:00.111260 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:00.111437 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.111544 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.111678 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260808 17:46:00.112438 9566 heartbeater.cc:347] Connected to a master server at 127.29.62.254:43633
I20260808 17:46:00.112522 9566 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:00.112715 9566 heartbeater.cc:511] Master 127.29.62.254:43633 requested a full tablet report, sending...
I20260808 17:46:00.113291 9229 ts_manager.cc:194] Registered new tserver with Master: 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969)
I20260808 17:46:00.113989 9229 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:46293
I20260808 17:46:00.114192 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 9343
I20260808 17:46:00.114312 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:46:00.114574 29947 external_mini_cluster.cc:1428] Setting key c631d4bd908fc74ab19736c61a343a02
I20260808 17:46:00.119871 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.113582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:50741 (local address 127.29.62.254:43633)
0808 17:46:00.113641 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.113642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.113693 (+ 51us) server_negotiation.cc:407] Connection header received
0808 17:46:00.113764 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.113765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.113781 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.113835 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.114208 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.114962 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.115697 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.116063 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.116182 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.117135 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:00.117137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:00.117137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:00.117145 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:00.118745 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.119078 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.119079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.119080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.119134 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.119351 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.119352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.119352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.119461 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:00.119534 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.119698 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.119782 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:46:00.119879 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.113517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43633 (local address 127.29.62.193:50741)
0808 17:46:00.113642 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.113653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.113709 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.113841 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.113842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.114043 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:00.114141 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.114143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.115067 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.115068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.115609 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.115611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.116185 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.116186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.116271 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.116712 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:00.116717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:00.117022 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:00.118827 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.118828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.118828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.119002 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.119208 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.119209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.119209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.119275 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.119527 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:00.119528 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:00.119574 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.119667 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.119774 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab.
Entry for principal oryx/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab.
I20260808 17:46:00.140913 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:43633
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab
--principal=oryx/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:00.234704 9580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:00.234994 9580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:00.235030 9580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:00.236690 9580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:00.236750 9580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:00.236788 9580 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:46:00.236821 9580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:46:00.239234 9580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab
--principal=oryx/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:43633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:00.239573 9580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:00.239913 9580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:00.247448 9590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:00.247455 9592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:00.247455 9595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:00.248251 9580 server_base.cc:1034] running on GCE node
I20260808 17:46:00.248482 9580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:00.248744 9580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:00.249948 9580 hybrid_clock.cc:648] HybridClock initialized: now 1786211160249927 us; error 34 us; skew 500 ppm
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211160, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:00.253710 9580 init.cc:377] Logged in from keytab as oryx/127.29.62.194@KRBTEST.COM (short username oryx)
I20260808 17:46:00.254324 9580 webserver.cc:460] Webserver started at http://127.29.62.194:33613/ using document root <none> and password file <none>
I20260808 17:46:00.254666 9580 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:00.254732 9580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:00.254874 9580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:00.256372 9580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "f4dd378675864937bb745d4ade778eaf"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "a486d246380bd3d4a32b678e1be689d2"
server_key_iv: "e6b1bd0bdae27edfda280be525390294"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.256831 9580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "f4dd378675864937bb745d4ade778eaf"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "a486d246380bd3d4a32b678e1be689d2"
server_key_iv: "e6b1bd0bdae27edfda280be525390294"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.258816 9580 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 17:46:00.259714 9607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.259920 9580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:00.260030 9580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "f4dd378675864937bb745d4ade778eaf"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "a486d246380bd3d4a32b678e1be689d2"
server_key_iv: "e6b1bd0bdae27edfda280be525390294"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.260133 9580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:00.271090 9580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:00.273525 9580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:00.273697 9580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:00.274243 9580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:00.274310 9580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.274362 9580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:00.274416 9580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211160, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:46:00.282090 9580 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:33183
I20260808 17:46:00.282191 9815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:33183 every 8 connection(s)
I20260808 17:46:00.282559 9580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:46:00.285159 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.274856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:43253 (local address 127.29.62.254:43633)
0808 17:46:00.274937 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.274941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.275094 (+ 153us) server_negotiation.cc:407] Connection header received
0808 17:46:00.275627 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.275629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.275645 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.275728 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.276581 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.277414 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.278823 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.279036 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.279127 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.281804 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:00.281806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:00.281806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:00.281835 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:00.283764 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.284176 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.284177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.284177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.284235 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.284614 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.284614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.284614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.284721 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:00.284777 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.285030 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.285096 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:46:00.285318 9626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.274804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43633 (local address 127.29.62.194:43253)
0808 17:46:00.274961 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.275004 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.275528 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.275856 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.275862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.276180 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:00.276426 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.276432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.277537 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.277538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.278718 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.278722 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.279153 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.279154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.279287 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.279842 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:00.279859 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:00.281693 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:00.283853 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.283859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.283861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.284109 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.284333 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.284334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.284334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.284545 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.284788 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:00.284793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:00.284914 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.285000 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.285121 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260808 17:46:00.285842 9816 heartbeater.cc:347] Connected to a master server at 127.29.62.254:43633
I20260808 17:46:00.285943 9816 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:00.286157 9816 heartbeater.cc:511] Master 127.29.62.254:43633 requested a full tablet report, sending...
I20260808 17:46:00.286644 9229 ts_manager.cc:194] Registered new tserver with Master: f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183)
I20260808 17:46:00.287014 9229 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:43253
I20260808 17:46:00.287949 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 9580
I20260808 17:46:00.288084 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:46:00.288301 29947 external_mini_cluster.cc:1428] Setting key 8eacf86c1221f9fe89014da431cca3f8
I20260808 17:46:00.293118 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.287196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:60049 (local address 127.29.62.254:43633)
0808 17:46:00.287236 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.287238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.287277 (+ 39us) server_negotiation.cc:407] Connection header received
0808 17:46:00.287371 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.287371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.287382 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.287451 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.287771 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.288482 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.289283 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.289495 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.289580 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.290574 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:00.290575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:00.290575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:00.290582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:00.291938 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.292295 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.292296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.292297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.292352 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.292567 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.292568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.292568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.292715 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:00.292790 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.292953 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.293028 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:46:00.293133 9823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.287067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43633 (local address 127.29.62.194:60049)
0808 17:46:00.287218 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.287230 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.287293 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.287464 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.287464 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.287626 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:00.287686 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.287688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.288589 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.288590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.289198 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.289201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.289584 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.289585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.289660 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.290102 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:00.290107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:00.290457 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:00.292048 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.292049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.292049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.292222 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.292429 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.292430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.292430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.292502 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.292795 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:00.292796 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:00.292842 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.292926 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.293030 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab.
Entry for principal oryx/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab.
I20260808 17:46:00.314587 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:43633
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab
--principal=oryx/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:00.400427 9827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:00.400669 9827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:00.400696 9827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:00.401930 9827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:00.401981 9827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:00.402004 9827 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 17:46:00.402052 9827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:46:00.403832 9827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab
--principal=oryx/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.62.254:43633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:00.404107 9827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:00.404363 9827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:00.409438 9834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:00.409438 9836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:00.409499 9840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:00.409950 9827 server_base.cc:1034] running on GCE node
I20260808 17:46:00.410158 9827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:00.410521 9827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:00.411676 9827 hybrid_clock.cc:648] HybridClock initialized: now 1786211160411637 us; error 56 us; skew 500 ppm
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211160, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:00.414441 9827 init.cc:377] Logged in from keytab as oryx/127.29.62.195@KRBTEST.COM (short username oryx)
I20260808 17:46:00.414935 9827 webserver.cc:460] Webserver started at http://127.29.62.195:33561/ using document root <none> and password file <none>
I20260808 17:46:00.415174 9827 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:00.415242 9827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:00.415402 9827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:00.416499 9827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "1826fffa7ee54d4393a3a351c48cd8b8"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "1991d6e87d2a28e0eb6a25e9aec9be99"
server_key_iv: "a276b853a13d174ca0a27fb8ea3d12f8"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.416899 9827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "1826fffa7ee54d4393a3a351c48cd8b8"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "1991d6e87d2a28e0eb6a25e9aec9be99"
server_key_iv: "a276b853a13d174ca0a27fb8ea3d12f8"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.418315 9827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:46:00.419149 9848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.419329 9827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:00.419441 9827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "1826fffa7ee54d4393a3a351c48cd8b8"
format_stamp: "Formatted at 2026-08-08 17:46:00 on dist-test-slave-tk9z"
server_key: "1991d6e87d2a28e0eb6a25e9aec9be99"
server_key_iv: "a276b853a13d174ca0a27fb8ea3d12f8"
server_key_version: "encryptionkey@0"
I20260808 17:46:00.419528 9827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:00.429334 9827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:00.431782 9827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:00.431946 9827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:00.432521 9827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:00.432582 9827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.432626 9827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:00.432654 9827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211160, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:46:00.440205 9827 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:37561
I20260808 17:46:00.440310 10031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:37561 every 8 connection(s)
I20260808 17:46:00.440709 9827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:46:00.442849 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.433060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:45151 (local address 127.29.62.254:43633)
0808 17:46:00.433119 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.433121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.433321 (+ 200us) server_negotiation.cc:407] Connection header received
0808 17:46:00.433831 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.433833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.433850 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.433925 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.434663 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.435505 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.436594 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.436804 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.436890 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.439781 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:00.439782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:00.439783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:00.439791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:00.441540 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.441933 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.441935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.441935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.442002 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.442234 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.442234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.442235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.442351 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:00.442475 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.442703 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.442772 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:46:00.442993 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.433050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43633 (local address 127.29.62.195:45151)
0808 17:46:00.433203 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.433237 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.433723 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.433995 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.433999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.434318 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:00.434578 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.434584 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.435638 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.435639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.436494 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.436497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.436903 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.436904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.437037 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.437572 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:00.437581 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:00.439684 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:00.441615 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.441619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.441621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.441854 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.442074 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.442075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.442075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.442172 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.442462 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:00.442466 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:00.442587 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.442671 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.442795 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260808 17:46:00.443446 10032 heartbeater.cc:347] Connected to a master server at 127.29.62.254:43633
I20260808 17:46:00.443526 10032 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:00.443717 10032 heartbeater.cc:511] Master 127.29.62.254:43633 requested a full tablet report, sending...
I20260808 17:46:00.444170 9228 ts_manager.cc:194] Registered new tserver with Master: 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561)
I20260808 17:46:00.444684 9228 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:45151
I20260808 17:46:00.449862 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.444549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:49227 (local address 127.29.62.254:43633)
0808 17:46:00.444588 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.444590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.444696 (+ 106us) server_negotiation.cc:407] Connection header received
0808 17:46:00.444769 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.444770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.444780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.444827 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.445183 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.445757 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.446494 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.446671 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.446723 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.447630 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:00.447631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:00.447631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:00.447636 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:00.448947 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.449269 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.449270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.449270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.449320 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.449494 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.449495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.449495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.449588 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:00.449639 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.449743 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.449800 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:46:00.449882 10041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.444491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43633 (local address 127.29.62.195:49227)
0808 17:46:00.444625 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.444635 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.444711 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.444872 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.444873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.445047 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:00.445106 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.445107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.445852 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.445852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.446355 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.446357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.446732 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.446733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.446785 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.447235 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:00.447239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:00.447551 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:00.449028 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.449029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.449030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.449204 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.449376 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.449377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.449377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.449439 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.449640 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:00.449641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:00.449679 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.449729 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.449808 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260808 17:46:00.450328 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 9827
I20260808 17:46:00.450491 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:46:00.450721 29947 external_mini_cluster.cc:1428] Setting key 33bbfcc2570002cac1400fc384e394b3
I20260808 17:46:00.451774 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211160, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 17:46:00.474624 9333 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:00.469100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46734 (local address 127.29.62.254:43633)
0808 17:46:00.469176 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.469178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.469185 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:46:00.469210 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.469211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.469220 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.469287 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.469874 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.470445 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.471201 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.471395 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.471504 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.474524 (+ 3020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46734: BlockingRecv error: recv got EOF from 127.0.0.1:46734 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211160, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:46:00.481797 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.475462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.29.62.254:43633)
0808 17:46:00.475520 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.475522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.475549 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:46:00.475602 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.475602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.475611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.475669 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.476093 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.476624 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.477255 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.477449 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.477499 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.479551 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:00.479553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:00.479553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:00.479560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:00.480936 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.481229 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.481230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.481230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.481281 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.481433 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.481433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.481434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.481527 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:00.481599 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.481677 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.481734 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:46:00.483508 9228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:46:00.485239 9228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:46:00.491690 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.487703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33183 (local address 127.0.0.1:45686)
0808 17:46:00.488024 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.488036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.488101 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.488357 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.488360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.488362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:00.488435 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.488437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.489758 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.489759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.490668 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.490671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.491375 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.491376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.491467 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.491481 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.491556 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.491615 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":33,"thread_start_us":87,"threads_started":1}
I20260808 17:46:00.491734 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.487834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.29.62.194:33183)
0808 17:46:00.488040 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.488045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.488094 (+ 49us) server_negotiation.cc:407] Connection header received
0808 17:46:00.488172 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.488175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.488192 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.488269 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:00.488523 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.489648 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.490761 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.491289 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.491366 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.491442 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.491585 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.491628 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
I20260808 17:46:00.492015 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.487911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:35969 (local address 127.0.0.1:50054)
0808 17:46:00.488091 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.488101 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.488147 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.488546 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.488547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.488550 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:00.488599 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.488601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.489449 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.489450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.490719 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.490722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.491755 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.491756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.491829 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.491833 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.491888 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.491931 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260808 17:46:00.492367 10076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.488139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.29.62.193:35969)
0808 17:46:00.488311 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.488316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.488327 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:46:00.488368 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.488371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.488387 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.488475 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:00.488660 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.489344 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.490807 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.491589 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.491647 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.491719 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.492190 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.492239 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260808 17:46:00.492475 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.487693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37561 (local address 127.0.0.1:45114)
0808 17:46:00.487860 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.487904 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.487970 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.488520 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.488520 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.488522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:00.488566 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.488568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.489772 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.489773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.491202 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.491205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.492250 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.492252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.492313 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.492317 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.492367 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.492402 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":27,"thread_start_us":62,"threads_started":1}
I20260808 17:46:00.492574 9706 tablet_service.cc:1467] Processing CreateTablet for tablet 318aa534aab94cf68b1d826633f23f69 (DEFAULT_TABLE table=test-table [id=6fdccef611494ac7ad2ff3addd849ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:00.492808 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.487799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.29.62.195:37561)
0808 17:46:00.488270 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.488275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.488306 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:46:00.488347 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.488349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.488364 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.488438 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:00.488644 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.489690 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.491297 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.492066 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.492137 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.492213 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.492663 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.492708 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":80,"threads_started":1}
I20260808 17:46:00.492987 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318aa534aab94cf68b1d826633f23f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.492990 9463 tablet_service.cc:1467] Processing CreateTablet for tablet 318aa534aab94cf68b1d826633f23f69 (DEFAULT_TABLE table=test-table [id=6fdccef611494ac7ad2ff3addd849ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:00.493371 9463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318aa534aab94cf68b1d826633f23f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.493422 9933 tablet_service.cc:1467] Processing CreateTablet for tablet 318aa534aab94cf68b1d826633f23f69 (DEFAULT_TABLE table=test-table [id=6fdccef611494ac7ad2ff3addd849ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:00.493739 9933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318aa534aab94cf68b1d826633f23f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.496192 10077 tablet_bootstrap.cc:492] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Bootstrap starting.
I20260808 17:46:00.496857 10077 tablet_bootstrap.cc:654] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.497329 10079 tablet_bootstrap.cc:492] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Bootstrap starting.
I20260808 17:46:00.497223 10077 log.cc:824] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:00.497982 10079 tablet_bootstrap.cc:654] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.498272 10077 tablet_bootstrap.cc:492] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: No bootstrap required, opened a new log
I20260808 17:46:00.498312 10079 log.cc:824] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:00.498348 10077 ts_tablet_manager.cc:1397] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:46:00.498545 10080 tablet_bootstrap.cc:492] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Bootstrap starting.
I20260808 17:46:00.499166 10080 tablet_bootstrap.cc:654] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.499449 10079 tablet_bootstrap.cc:492] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: No bootstrap required, opened a new log
I20260808 17:46:00.499465 10080 log.cc:824] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:00.499532 10079 ts_tablet_manager.cc:1397] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:46:00.500123 10080 tablet_bootstrap.cc:492] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: No bootstrap required, opened a new log
I20260808 17:46:00.500202 10080 ts_tablet_manager.cc:1397] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:46:00.500494 10077 raft_consensus.cc:348] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.500672 10077 raft_consensus.cc:374] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.500710 10077 raft_consensus.cc:729] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4dd378675864937bb745d4ade778eaf, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.500836 10077 consensus_queue.cc:260] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.501137 10077 ts_tablet_manager.cc:1428] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:46:00.501183 9816 heartbeater.cc:503] Master 127.29.62.254:43633 was elected leader, sending a full tablet report...
I20260808 17:46:00.501566 10079 raft_consensus.cc:348] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.501681 10079 raft_consensus.cc:374] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.501701 10079 raft_consensus.cc:729] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b5658b628e47bfb8c7a8476c38ac60, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.501801 10079 consensus_queue.cc:260] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.502043 10079 ts_tablet_manager.cc:1428] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:46:00.502070 9566 heartbeater.cc:503] Master 127.29.62.254:43633 was elected leader, sending a full tablet report...
I20260808 17:46:00.502487 10080 raft_consensus.cc:348] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.502650 10080 raft_consensus.cc:374] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.502701 10080 raft_consensus.cc:729] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1826fffa7ee54d4393a3a351c48cd8b8, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.502812 10080 consensus_queue.cc:260] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.503121 10080 ts_tablet_manager.cc:1428] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:46:00.503206 10032 heartbeater.cc:503] Master 127.29.62.254:43633 was elected leader, sending a full tablet report...
W20260808 17:46:00.533195 9818 tablet.cc:2369] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:46:00.610697 9568 tablet.cc:2369] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:46:00.691354 10035 tablet.cc:2369] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:46:00.704599 10088 raft_consensus.cc:482] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:46:00.704739 10088 raft_consensus.cc:504] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.705173 10088 leader_election.cc:288] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183), 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969)
I20260808 17:46:00.708339 10076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.705678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:45903 (local address 127.29.62.193:35969)
0808 17:46:00.705762 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.705781 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.705795 (+ 14us) server_negotiation.cc:407] Connection header received
0808 17:46:00.705877 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.705878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.705888 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.705952 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:00.706145 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.706727 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.707634 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.707978 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.708073 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.708083 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.708218 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.708260 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 17:46:00.708339 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.705433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33183 (local address 127.29.62.195:50311)
0808 17:46:00.705516 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.705532 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.705608 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.705839 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.705841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.705843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:00.705927 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.705929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.706682 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.706683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.707496 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.707499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.708045 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.708047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.708150 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.708153 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.708225 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.708264 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 17:46:00.708341 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.705540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:50311 (local address 127.29.62.194:33183)
0808 17:46:00.705594 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.705597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.705608 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:46:00.705715 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.705718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.705729 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.705793 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:00.706017 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.706597 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.707588 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.707963 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.708019 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.708027 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.708241 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.708279 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:46:00.708348 10090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.705577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:35969 (local address 127.29.62.195:45903)
0808 17:46:00.705750 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.705760 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.705804 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.705961 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.705962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.705982 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:00.706069 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.706071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.706802 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.706803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.707523 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.707525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.708083 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.708084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.708145 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.708148 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.708190 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.708250 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260808 17:46:00.708725 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318aa534aab94cf68b1d826633f23f69" candidate_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4dd378675864937bb745d4ade778eaf" is_pre_election: true
I20260808 17:46:00.708729 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318aa534aab94cf68b1d826633f23f69" candidate_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60" is_pre_election: true
I20260808 17:46:00.708897 9736 raft_consensus.cc:2454] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1826fffa7ee54d4393a3a351c48cd8b8 in term 0.
I20260808 17:46:00.708909 9493 raft_consensus.cc:2454] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1826fffa7ee54d4393a3a351c48cd8b8 in term 0.
I20260808 17:46:00.709137 9855 leader_election.cc:302] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1826fffa7ee54d4393a3a351c48cd8b8, 18b5658b628e47bfb8c7a8476c38ac60; no voters:
I20260808 17:46:00.709263 10088 raft_consensus.cc:2781] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:46:00.709344 10088 raft_consensus.cc:482] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:46:00.709381 10088 raft_consensus.cc:3037] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:00.710417 10088 raft_consensus.cc:504] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.710568 10088 leader_election.cc:288] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [CANDIDATE]: Term 1 election: Requested vote from peers f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183), 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969)
I20260808 17:46:00.710743 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318aa534aab94cf68b1d826633f23f69" candidate_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4dd378675864937bb745d4ade778eaf"
I20260808 17:46:00.710785 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318aa534aab94cf68b1d826633f23f69" candidate_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60"
I20260808 17:46:00.710834 9736 raft_consensus.cc:3037] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:00.710889 9493 raft_consensus.cc:3037] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:00.711585 9736 raft_consensus.cc:2454] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1826fffa7ee54d4393a3a351c48cd8b8 in term 1.
I20260808 17:46:00.711587 9493 raft_consensus.cc:2454] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1826fffa7ee54d4393a3a351c48cd8b8 in term 1.
I20260808 17:46:00.711782 9855 leader_election.cc:302] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1826fffa7ee54d4393a3a351c48cd8b8, 18b5658b628e47bfb8c7a8476c38ac60; no voters:
I20260808 17:46:00.711884 10088 raft_consensus.cc:2781] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:00.712018 10088 raft_consensus.cc:686] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 LEADER]: Becoming Leader. State: Replica: 1826fffa7ee54d4393a3a351c48cd8b8, State: Running, Role: LEADER
I20260808 17:46:00.712145 10088 consensus_queue.cc:237] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } }
I20260808 17:46:00.713007 9229 catalog_manager.cc:5495] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195). New cstate: current_term: 1 leader_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } health_report { overall_health: UNKNOWN } } }
I20260808 17:46:00.774447 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.771960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.29.62.195:37561)
0808 17:46:00.772031 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.772033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.772042 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:00.772090 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.772090 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.772109 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.772193 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:46:00.772342 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.772830 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.773609 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.773803 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.773882 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.774069 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:46:00.774172 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:46:00.774260 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.774304 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.774325 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:46:00.776501 9493 raft_consensus.cc:1261] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Refusing update from remote peer 1826fffa7ee54d4393a3a351c48cd8b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:00.776501 9736 raft_consensus.cc:1261] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Refusing update from remote peer 1826fffa7ee54d4393a3a351c48cd8b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:00.776846 10091 consensus_queue.cc:1035] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:00.777150 10088 consensus_queue.cc:1035] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:00.779774 10100 mvcc.cc:205] Tried to move back new op lower bound from 7316320914537185280 to 7316320914277453824. Current Snapshot: MvccSnapshot[applied={T|T < 7316320914537185280 or (T in {7316320914537185280})}]
I20260808 17:46:00.780895 10104 mvcc.cc:205] Tried to move back new op lower bound from 7316320914537185280 to 7316320914277453824. Current Snapshot: MvccSnapshot[applied={T|T < 7316320914537185280}]
I20260808 17:46:00.782156 9228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:00.782263 9228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:00.783053 9228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: Sending DeleteTablet for 3 replicas of tablet 318aa534aab94cf68b1d826633f23f69
I20260808 17:46:00.783424 9933 tablet_service.cc:1514] Processing DeleteTablet for tablet 318aa534aab94cf68b1d826633f23f69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:00 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45114
I20260808 17:46:00.783406 9706 tablet_service.cc:1514] Processing DeleteTablet for tablet 318aa534aab94cf68b1d826633f23f69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:00 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45686
I20260808 17:46:00.783653 9463 tablet_service.cc:1514] Processing DeleteTablet for tablet 318aa534aab94cf68b1d826633f23f69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:00 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:50054
I20260808 17:46:00.783941 10119 tablet_replica.cc:330] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: stopping tablet replica
I20260808 17:46:00.783967 10120 tablet_replica.cc:330] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: stopping tablet replica
I20260808 17:46:00.783957 9228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 17:46:00.784062 10120 raft_consensus.cc:2229] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:46:00.784101 10119 raft_consensus.cc:2229] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 17:46:00.784099 9228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:46:00.784158 10120 raft_consensus.cc:2258] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:00.784184 10119 raft_consensus.cc:2258] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:00.784648 10119 ts_tablet_manager.cc:1905] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:00.784648 10120 ts_tablet_manager.cc:1905] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:00.785989 9463 tablet_service.cc:1467] Processing CreateTablet for tablet 759c0d9273cb4a33b24dd5526c1de959 (DEFAULT_TABLE table=test-table [id=85cb6a7980ea46a7b6323ccae1e6195a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:00.786141 9463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759c0d9273cb4a33b24dd5526c1de959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.786329 10119 ts_tablet_manager.cc:1918] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:46:00.786401 10120 ts_tablet_manager.cc:1918] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:46:00.786427 10119 log.cc:1196] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/318aa534aab94cf68b1d826633f23f69
I20260808 17:46:00.786463 10120 log.cc:1196] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/318aa534aab94cf68b1d826633f23f69
I20260808 17:46:00.786463 9706 tablet_service.cc:1467] Processing CreateTablet for tablet 759c0d9273cb4a33b24dd5526c1de959 (DEFAULT_TABLE table=test-table [id=85cb6a7980ea46a7b6323ccae1e6195a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:00.786574 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759c0d9273cb4a33b24dd5526c1de959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.786715 10120 ts_tablet_manager.cc:1939] T 318aa534aab94cf68b1d826633f23f69 P 18b5658b628e47bfb8c7a8476c38ac60: Deleting consensus metadata
I20260808 17:46:00.786698 10119 ts_tablet_manager.cc:1939] T 318aa534aab94cf68b1d826633f23f69 P f4dd378675864937bb745d4ade778eaf: Deleting consensus metadata
I20260808 17:46:00.787111 10117 tablet_replica.cc:330] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: stopping tablet replica
I20260808 17:46:00.787182 10117 raft_consensus.cc:2229] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:46:00.787248 10117 raft_consensus.cc:2258] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:00.787760 10077 tablet_bootstrap.cc:492] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: Bootstrap starting.
I20260808 17:46:00.788026 10117 ts_tablet_manager.cc:1905] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:00.788039 9209 catalog_manager.cc:4841] TS f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183): tablet 318aa534aab94cf68b1d826633f23f69 (table test-table [id=6fdccef611494ac7ad2ff3addd849ff5]) successfully deleted
I20260808 17:46:00.788141 9209 catalog_manager.cc:4841] TS 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969): tablet 318aa534aab94cf68b1d826633f23f69 (table test-table [id=6fdccef611494ac7ad2ff3addd849ff5]) successfully deleted
I20260808 17:46:00.788182 10077 tablet_bootstrap.cc:654] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.788618 9933 tablet_service.cc:1467] Processing CreateTablet for tablet 759c0d9273cb4a33b24dd5526c1de959 (DEFAULT_TABLE table=test-table [id=85cb6a7980ea46a7b6323ccae1e6195a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:00.788733 9933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759c0d9273cb4a33b24dd5526c1de959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.789677 10079 tablet_bootstrap.cc:492] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: Bootstrap starting.
I20260808 17:46:00.789688 10117 ts_tablet_manager.cc:1918] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:46:00.789750 10117 log.cc:1196] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/318aa534aab94cf68b1d826633f23f69
I20260808 17:46:00.790007 10117 ts_tablet_manager.cc:1939] T 318aa534aab94cf68b1d826633f23f69 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting consensus metadata
I20260808 17:46:00.790011 10080 tablet_bootstrap.cc:492] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: Bootstrap starting.
I20260808 17:46:00.790094 10079 tablet_bootstrap.cc:654] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.790529 10080 tablet_bootstrap.cc:654] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.790922 9208 catalog_manager.cc:4841] TS 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561): tablet 318aa534aab94cf68b1d826633f23f69 (table test-table [id=6fdccef611494ac7ad2ff3addd849ff5]) successfully deleted
I20260808 17:46:00.791023 10079 tablet_bootstrap.cc:492] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: No bootstrap required, opened a new log
I20260808 17:46:00.791065 10079 ts_tablet_manager.cc:1397] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:00.791137 10080 tablet_bootstrap.cc:492] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: No bootstrap required, opened a new log
I20260808 17:46:00.791168 10080 ts_tablet_manager.cc:1397] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:00.791196 10079 raft_consensus.cc:348] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.791260 10079 raft_consensus.cc:374] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.791281 10079 raft_consensus.cc:729] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b5658b628e47bfb8c7a8476c38ac60, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.791270 10080 raft_consensus.cc:348] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.791316 10080 raft_consensus.cc:374] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.791332 10079 consensus_queue.cc:260] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.791363 10080 raft_consensus.cc:729] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1826fffa7ee54d4393a3a351c48cd8b8, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.791409 10080 consensus_queue.cc:260] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.791445 10079 ts_tablet_manager.cc:1428] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.791513 10080 ts_tablet_manager.cc:1428] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.792521 10077 tablet_bootstrap.cc:492] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: No bootstrap required, opened a new log
I20260808 17:46:00.792565 10077 ts_tablet_manager.cc:1397] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 17:46:00.792680 10077 raft_consensus.cc:348] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.792733 10077 raft_consensus.cc:374] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.792747 10077 raft_consensus.cc:729] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4dd378675864937bb745d4ade778eaf, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.792805 10077 consensus_queue.cc:260] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.792903 10077 ts_tablet_manager.cc:1428] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.905040 10112 raft_consensus.cc:482] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:46:00.905143 10112 raft_consensus.cc:504] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.905303 10112 leader_election.cc:288] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969), f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183)
I20260808 17:46:00.905614 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759c0d9273cb4a33b24dd5526c1de959" candidate_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4dd378675864937bb745d4ade778eaf" is_pre_election: true
I20260808 17:46:00.905723 9736 raft_consensus.cc:2454] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1826fffa7ee54d4393a3a351c48cd8b8 in term 0.
I20260808 17:46:00.905838 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759c0d9273cb4a33b24dd5526c1de959" candidate_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60" is_pre_election: true
I20260808 17:46:00.905925 9855 leader_election.cc:302] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1826fffa7ee54d4393a3a351c48cd8b8, f4dd378675864937bb745d4ade778eaf; no voters:
I20260808 17:46:00.905964 9493 raft_consensus.cc:2454] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1826fffa7ee54d4393a3a351c48cd8b8 in term 0.
I20260808 17:46:00.906008 10112 raft_consensus.cc:2781] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:46:00.906046 10112 raft_consensus.cc:482] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:46:00.906075 10112 raft_consensus.cc:3037] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:00.907079 10112 raft_consensus.cc:504] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.907217 10112 leader_election.cc:288] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [CANDIDATE]: Term 1 election: Requested vote from peers 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969), f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183)
I20260808 17:46:00.907423 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759c0d9273cb4a33b24dd5526c1de959" candidate_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60"
I20260808 17:46:00.907416 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759c0d9273cb4a33b24dd5526c1de959" candidate_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4dd378675864937bb745d4ade778eaf"
I20260808 17:46:00.907518 9493 raft_consensus.cc:3037] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:00.907536 9736 raft_consensus.cc:3037] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:00.908433 9493 raft_consensus.cc:2454] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1826fffa7ee54d4393a3a351c48cd8b8 in term 1.
I20260808 17:46:00.908545 9736 raft_consensus.cc:2454] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1826fffa7ee54d4393a3a351c48cd8b8 in term 1.
I20260808 17:46:00.908631 9855 leader_election.cc:302] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1826fffa7ee54d4393a3a351c48cd8b8, 18b5658b628e47bfb8c7a8476c38ac60; no voters:
I20260808 17:46:00.908772 10112 raft_consensus.cc:2781] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:00.908826 10112 raft_consensus.cc:686] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 LEADER]: Becoming Leader. State: Replica: 1826fffa7ee54d4393a3a351c48cd8b8, State: Running, Role: LEADER
I20260808 17:46:00.908883 10112 consensus_queue.cc:237] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.909494 9228 catalog_manager.cc:5495] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195). New cstate: current_term: 1 leader_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } health_report { overall_health: HEALTHY } } }
Aug 08 17:46:00 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211159, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.254@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:46:00.977909 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.971246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46760 (local address 127.29.62.254:43633)
0808 17:46:00.971317 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:00.971320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:00.971453 (+ 133us) server_negotiation.cc:407] Connection header received
0808 17:46:00.971520 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:00.971521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:00.971530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:00.971572 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:00.972036 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.972516 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.973167 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.973342 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.973394 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.975566 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:00.975567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:00.975567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:00.975574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:00.976989 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.977313 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.977314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.977314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.977364 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:00.977525 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:00.977526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:00.977526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:00.977617 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:00.977663 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:00.977778 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 17:46:00.977845 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:46:00.978091 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:00.971185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:43633 (local address 127.0.0.1:46760)
0808 17:46:00.971359 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:00.971369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:00.971460 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:00.971599 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:00.971600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:00.971913 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:00.971971 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.971973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.972585 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.972586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.973100 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:00.973102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:00.973414 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:00.973415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:00.973456 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:00.973932 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:00.973937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:00.975489 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:00.977055 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.977061 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.977063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.977252 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.977423 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:00.977424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:00.977424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:00.977480 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:00.977667 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:00.977673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:00.977712 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:46:00.977762 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:46:00.977964 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":63,"thread_start_us":102,"threads_started":1}
I20260808 17:46:00.979971 9228 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:46760:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 17:46:00.980191 9228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:46:00.981738 9933 tablet_service.cc:1467] Processing CreateTablet for tablet f2bf6179b7d3423f98687ab130e575cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91b30089df2b45a38989dd485201996a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:46:00.981738 9463 tablet_service.cc:1467] Processing CreateTablet for tablet f2bf6179b7d3423f98687ab130e575cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91b30089df2b45a38989dd485201996a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:46:00.981750 9706 tablet_service.cc:1467] Processing CreateTablet for tablet f2bf6179b7d3423f98687ab130e575cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91b30089df2b45a38989dd485201996a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 17:46:00.981917 9933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2bf6179b7d3423f98687ab130e575cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.981953 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2bf6179b7d3423f98687ab130e575cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.981988 9463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2bf6179b7d3423f98687ab130e575cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:00.983525 10080 tablet_bootstrap.cc:492] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: Bootstrap starting.
I20260808 17:46:00.983585 10077 tablet_bootstrap.cc:492] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: Bootstrap starting.
I20260808 17:46:00.983588 10079 tablet_bootstrap.cc:492] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: Bootstrap starting.
I20260808 17:46:00.983974 10077 tablet_bootstrap.cc:654] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.983984 10079 tablet_bootstrap.cc:654] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.983983 10080 tablet_bootstrap.cc:654] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:00.984644 10080 tablet_bootstrap.cc:492] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: No bootstrap required, opened a new log
I20260808 17:46:00.984716 10080 ts_tablet_manager.cc:1397] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:00.984736 10077 tablet_bootstrap.cc:492] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: No bootstrap required, opened a new log
I20260808 17:46:00.984772 10077 ts_tablet_manager.cc:1397] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:00.984809 10079 tablet_bootstrap.cc:492] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: No bootstrap required, opened a new log
I20260808 17:46:00.984853 10079 ts_tablet_manager.cc:1397] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:00.984835 10080 raft_consensus.cc:348] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.984913 10080 raft_consensus.cc:374] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.984886 10077 raft_consensus.cc:348] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.984941 10077 raft_consensus.cc:374] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.984941 10080 raft_consensus.cc:729] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1826fffa7ee54d4393a3a351c48cd8b8, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.984956 10077 raft_consensus.cc:729] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4dd378675864937bb745d4ade778eaf, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.985002 10080 consensus_queue.cc:260] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.984987 10079 raft_consensus.cc:348] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.985025 10077 consensus_queue.cc:260] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.985064 10079 raft_consensus.cc:374] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:00.985086 10079 raft_consensus.cc:729] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b5658b628e47bfb8c7a8476c38ac60, State: Initialized, Role: FOLLOWER
I20260808 17:46:00.985105 10080 ts_tablet_manager.cc:1428] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:00.985139 10077 ts_tablet_manager.cc:1428] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:00.985126 10079 consensus_queue.cc:260] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:00.985209 10079 ts_tablet_manager.cc:1428] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 17:46:00.985184 10085 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:00.985210 10112 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:00.985280 10085 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:46:00.985284 10112 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:46:00.985432 10087 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:00.985554 10087 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:46:01.324801 10085 raft_consensus.cc:482] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:46:01.324906 10085 raft_consensus.cc:504] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.325291 10085 leader_election.cc:288] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969), 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561)
I20260808 17:46:01.328307 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.325497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:35969 (local address 127.29.62.194:33621)
0808 17:46:01.325634 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.325646 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:01.325705 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:01.325938 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:01.325939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:01.325942 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:01.326005 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.326006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.326757 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.326758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.327463 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.327465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.328072 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.328073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.328134 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.328141 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:46:01.328196 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:46:01.328225 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260808 17:46:01.328341 10139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.325556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:33621 (local address 127.29.62.193:35969)
0808 17:46:01.325774 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.325776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.325784 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:46:01.325813 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.325814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.325830 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.325911 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:01.326106 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.326657 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.327560 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.327994 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.328054 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.328062 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.328212 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.328259 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":56,"threads_started":1}
I20260808 17:46:01.328421 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.325502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37561 (local address 127.29.62.194:45169)
0808 17:46:01.325641 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.325652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:01.325712 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:01.325902 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:01.325903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:01.325906 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:01.325995 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.325997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.326665 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.326666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.327454 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.327456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.328148 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.328149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.328236 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.328240 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:46:01.328300 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:46:01.328338 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260808 17:46:01.328584 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2bf6179b7d3423f98687ab130e575cf" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60" is_pre_election: true
I20260808 17:46:01.328670 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.325596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:45169 (local address 127.29.62.195:37561)
0808 17:46:01.325705 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.325708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.325718 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:46:01.325811 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.325812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.325820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.325875 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:01.326073 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.326590 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.327555 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.328041 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.328118 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.328127 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.328320 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.328602 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260808 17:46:01.328711 9493 raft_consensus.cc:2454] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 0.
I20260808 17:46:01.328895 9620 leader_election.cc:302] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b5658b628e47bfb8c7a8476c38ac60, f4dd378675864937bb745d4ade778eaf; no voters:
I20260808 17:46:01.329059 10085 raft_consensus.cc:2781] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:46:01.328985 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2bf6179b7d3423f98687ab130e575cf" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" is_pre_election: true
I20260808 17:46:01.329098 10085 raft_consensus.cc:482] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:46:01.329125 10085 raft_consensus.cc:3037] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.329128 9964 raft_consensus.cc:2454] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 0.
I20260808 17:46:01.329865 10085 raft_consensus.cc:504] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.330008 10085 leader_election.cc:288] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 election: Requested vote from peers 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969), 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561)
I20260808 17:46:01.330132 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2bf6179b7d3423f98687ab130e575cf" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60"
I20260808 17:46:01.330150 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2bf6179b7d3423f98687ab130e575cf" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1826fffa7ee54d4393a3a351c48cd8b8"
I20260808 17:46:01.330229 9964 raft_consensus.cc:3037] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.330233 9493 raft_consensus.cc:3037] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.330914 9964 raft_consensus.cc:2454] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 1.
I20260808 17:46:01.330911 9493 raft_consensus.cc:2454] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 1.
I20260808 17:46:01.331101 9619 leader_election.cc:302] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1826fffa7ee54d4393a3a351c48cd8b8, f4dd378675864937bb745d4ade778eaf; no voters:
I20260808 17:46:01.331208 10085 raft_consensus.cc:2781] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:01.331332 10085 raft_consensus.cc:686] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [term 1 LEADER]: Becoming Leader. State: Replica: f4dd378675864937bb745d4ade778eaf, State: Running, Role: LEADER
I20260808 17:46:01.331400 10085 consensus_queue.cc:237] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.331728 10142 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: TxnStatusTablet state changed. Reason: New leader f4dd378675864937bb745d4ade778eaf. Latest consensus state: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:01.331818 10142 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:46:01.331979 10144 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 17:46:01.332100 10144 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 17:46:01.332175 9228 catalog_manager.cc:5495] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf reported cstate change: term changed from 0 to 1, leader changed from <none> to f4dd378675864937bb745d4ade778eaf (127.29.62.194). New cstate: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } health_report { overall_health: UNKNOWN } } }
I20260808 17:46:01.340443 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.337522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33183 (local address 127.0.0.1:45688)
0808 17:46:01.337596 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.337608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:01.337695 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:01.337869 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:01.337869 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:01.337871 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:46:01.337911 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.337913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.338886 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.338887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.339437 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.339439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.339987 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.339988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.340028 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.340043 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.340313 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.340320 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:46:01.340354 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 17:46:01.340392 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:46:01.340644 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.337598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45688 (local address 127.29.62.194:33183)
0808 17:46:01.337716 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.337718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.337727 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:01.337762 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.337763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.337780 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.337871 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:46:01.337953 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.338750 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.339522 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.339861 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.339966 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.340131 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.340244 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.340323 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.340358 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.340378 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260808 17:46:01.341639 9493 raft_consensus.cc:1261] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Refusing update from remote peer f4dd378675864937bb745d4ade778eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:01.341711 9964 raft_consensus.cc:1261] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Refusing update from remote peer f4dd378675864937bb745d4ade778eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:01.341874 10085 consensus_queue.cc:1035] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:01.342010 10142 consensus_queue.cc:1035] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:01.342854 10087 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: TxnStatusTablet state changed. Reason: New leader f4dd378675864937bb745d4ade778eaf. Latest consensus state: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:01.342957 10087 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:46:01.343017 10112 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: TxnStatusTablet state changed. Reason: New leader f4dd378675864937bb745d4ade778eaf. Latest consensus state: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:01.343127 10112 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:46:01.343670 10142 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:01.343750 10142 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:46:01.343909 10112 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:01.343982 10112 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P 1826fffa7ee54d4393a3a351c48cd8b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:46:01.344281 10142 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:01.344370 10142 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P f4dd378675864937bb745d4ade778eaf: This TxnStatusTablet replica's current role is: LEADER
I20260808 17:46:01.344746 10087 tablet_replica.cc:439] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } } }
I20260808 17:46:01.344833 10087 tablet_replica.cc:442] T f2bf6179b7d3423f98687ab130e575cf P 18b5658b628e47bfb8c7a8476c38ac60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 17:46:01.350083 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.347508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33183 (local address 127.29.62.195:38705)
0808 17:46:01.347679 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.347688 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:01.347740 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:01.347867 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:01.347867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:01.347868 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:46:01.347903 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.347905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.348711 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.348712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.349267 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.349269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.349722 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.349723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.349772 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.349785 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.349953 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.349954 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:46:01.349996 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:46:01.350032 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260808 17:46:01.350081 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.347569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:38705 (local address 127.29.62.194:33183)
0808 17:46:01.347608 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.347611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.347752 (+ 141us) server_negotiation.cc:407] Connection header received
0808 17:46:01.347779 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.347780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.347799 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.347862 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:46:01.347940 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.348641 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.349344 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.349644 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.349713 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.349860 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.349891 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.349944 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.350000 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.350019 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:46:01.352257 9736 raft_consensus.cc:1261] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Refusing update from remote peer 1826fffa7ee54d4393a3a351c48cd8b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:01.352279 9493 raft_consensus.cc:1261] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Refusing update from remote peer 1826fffa7ee54d4393a3a351c48cd8b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:01.352499 10091 consensus_queue.cc:1035] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:01.352594 10112 consensus_queue.cc:1035] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:01.353612 10153 mvcc.cc:205] Tried to move back new op lower bound from 7316320916897484800 to 7316320915082973184. Current Snapshot: MvccSnapshot[applied={T|T < 7316320916897484800}]
I20260808 17:46:01.361177 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.357576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:47053 (local address 127.29.62.195:37561)
0808 17:46:01.357613 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.357615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.358559 (+ 944us) server_negotiation.cc:407] Connection header received
0808 17:46:01.358617 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.358618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.358626 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.358670 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:46:01.358803 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.359357 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.360568 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.360744 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.360791 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.361008 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.361031 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.361064 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.361117 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.361133 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:46:01.361267 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.357540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37561 (local address 127.29.62.194:47053)
0808 17:46:01.358488 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.358500 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:01.358561 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:01.358673 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:01.358674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:01.358685 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:46:01.358751 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.358752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.359518 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.359519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.360481 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.360483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.360843 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.360844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.360908 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.360924 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.361067 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.361068 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 17:46:01.361116 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 17:46:01.361156 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":86,"threads_started":1}
I20260808 17:46:01.368798 9227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:01.368918 9227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:01.369598 9227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: Sending DeleteTablet for 3 replicas of tablet 759c0d9273cb4a33b24dd5526c1de959
I20260808 17:46:01.369889 9706 tablet_service.cc:1514] Processing DeleteTablet for tablet 759c0d9273cb4a33b24dd5526c1de959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45686
I20260808 17:46:01.369915 9463 tablet_service.cc:1514] Processing DeleteTablet for tablet 759c0d9273cb4a33b24dd5526c1de959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:50054
I20260808 17:46:01.370092 9933 tablet_service.cc:1514] Processing DeleteTablet for tablet 759c0d9273cb4a33b24dd5526c1de959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45114
I20260808 17:46:01.370265 10181 tablet_replica.cc:330] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: stopping tablet replica
I20260808 17:46:01.370312 10181 raft_consensus.cc:2229] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:46:01.370385 10181 raft_consensus.cc:2258] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:01.370456 10176 tablet_replica.cc:330] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: stopping tablet replica
I20260808 17:46:01.370502 10177 tablet_replica.cc:330] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: stopping tablet replica
I20260808 17:46:01.370519 10176 raft_consensus.cc:2229] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:46:01.370549 10177 raft_consensus.cc:2229] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:46:01.370558 10176 raft_consensus.cc:2258] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:01.370589 10177 raft_consensus.cc:2258] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:01.370824 10177 ts_tablet_manager.cc:1905] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:01.370826 10176 ts_tablet_manager.cc:1905] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:01.372473 10177 ts_tablet_manager.cc:1918] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:46:01.372525 10177 log.cc:1196] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/759c0d9273cb4a33b24dd5526c1de959
I20260808 17:46:01.372748 10177 ts_tablet_manager.cc:1939] T 759c0d9273cb4a33b24dd5526c1de959 P 18b5658b628e47bfb8c7a8476c38ac60: Deleting consensus metadata
I20260808 17:46:01.372876 10176 ts_tablet_manager.cc:1918] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:46:01.372932 10176 log.cc:1196] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/759c0d9273cb4a33b24dd5526c1de959
I20260808 17:46:01.373035 9209 catalog_manager.cc:4841] TS 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969): tablet 759c0d9273cb4a33b24dd5526c1de959 (table test-table [id=85cb6a7980ea46a7b6323ccae1e6195a]) successfully deleted
I20260808 17:46:01.373178 10176 ts_tablet_manager.cc:1939] T 759c0d9273cb4a33b24dd5526c1de959 P f4dd378675864937bb745d4ade778eaf: Deleting consensus metadata
I20260808 17:46:01.373301 10181 ts_tablet_manager.cc:1905] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:01.373494 9209 catalog_manager.cc:4841] TS f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183): tablet 759c0d9273cb4a33b24dd5526c1de959 (table test-table [id=85cb6a7980ea46a7b6323ccae1e6195a]) successfully deleted
I20260808 17:46:01.374907 10181 ts_tablet_manager.cc:1918] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:46:01.374970 10181 log.cc:1196] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/759c0d9273cb4a33b24dd5526c1de959
I20260808 17:46:01.375186 10181 ts_tablet_manager.cc:1939] T 759c0d9273cb4a33b24dd5526c1de959 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting consensus metadata
I20260808 17:46:01.375491 9208 catalog_manager.cc:4841] TS 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561): tablet 759c0d9273cb4a33b24dd5526c1de959 (table test-table [id=85cb6a7980ea46a7b6323ccae1e6195a]) successfully deleted
Aug 08 17:46:01 dist-test-slave-tk9z krb5kdc[9158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 17:46:01 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:01 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 17:46:01.397284 9333 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:01.391875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.29.62.254:43633)
0808 17:46:01.391938 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.391941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.392047 (+ 106us) server_negotiation.cc:407] Connection header received
0808 17:46:01.392153 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.392154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.392167 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.392239 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:01.392827 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.393328 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.394077 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.394275 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.394353 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.397209 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46776: BlockingRecv error: recv got EOF from 127.0.0.1:46776 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 17:46:01 dist-test-slave-tk9z krb5kdc[9158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM
I20260808 17:46:01.405267 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.398181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.29.62.254:43633)
0808 17:46:01.398228 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.398231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.398253 (+ 22us) server_negotiation.cc:407] Connection header received
0808 17:46:01.398316 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.398317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.398326 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.398428 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:01.399029 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.399564 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.400224 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.400425 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.400502 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.402714 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:01.402716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:01.402716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:01.402724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:01.404100 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:01.404453 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:01.404454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:01.404454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:01.404518 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:01.404734 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:01.404735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:01.404735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:01.404854 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:01.404961 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.405075 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.405158 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:46:01.406929 9227 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:46:01.407119 9227 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:46:01.408576 9706 tablet_service.cc:1467] Processing CreateTablet for tablet e2d40abacb5540a982c0a3ae0e0e4cde (DEFAULT_TABLE table=test-table [id=3bfcb76466e24d479d18c52871ee7306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:01.408610 9933 tablet_service.cc:1467] Processing CreateTablet for tablet e2d40abacb5540a982c0a3ae0e0e4cde (DEFAULT_TABLE table=test-table [id=3bfcb76466e24d479d18c52871ee7306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:01.408746 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d40abacb5540a982c0a3ae0e0e4cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:01.408749 9933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d40abacb5540a982c0a3ae0e0e4cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:01.409771 10077 tablet_bootstrap.cc:492] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: Bootstrap starting.
I20260808 17:46:01.410143 9463 tablet_service.cc:1467] Processing CreateTablet for tablet e2d40abacb5540a982c0a3ae0e0e4cde (DEFAULT_TABLE table=test-table [id=3bfcb76466e24d479d18c52871ee7306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:01.410195 10077 tablet_bootstrap.cc:654] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:01.410286 9463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d40abacb5540a982c0a3ae0e0e4cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:01.410881 10077 tablet_bootstrap.cc:492] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: No bootstrap required, opened a new log
I20260808 17:46:01.410948 10077 ts_tablet_manager.cc:1397] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:01.411108 10077 raft_consensus.cc:348] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.411191 10077 raft_consensus.cc:374] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:01.411219 10077 raft_consensus.cc:729] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4dd378675864937bb745d4ade778eaf, State: Initialized, Role: FOLLOWER
I20260808 17:46:01.411257 10077 consensus_queue.cc:260] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.411356 10077 ts_tablet_manager.cc:1428] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:01.411602 10079 tablet_bootstrap.cc:492] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: Bootstrap starting.
I20260808 17:46:01.411630 10080 tablet_bootstrap.cc:492] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: Bootstrap starting.
I20260808 17:46:01.411947 10080 tablet_bootstrap.cc:654] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:01.412026 10079 tablet_bootstrap.cc:654] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:01.412930 10080 tablet_bootstrap.cc:492] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: No bootstrap required, opened a new log
I20260808 17:46:01.412946 10079 tablet_bootstrap.cc:492] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: No bootstrap required, opened a new log
I20260808 17:46:01.412985 10080 ts_tablet_manager.cc:1397] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:01.413007 10079 ts_tablet_manager.cc:1397] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:01.413127 10080 raft_consensus.cc:348] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.413206 10080 raft_consensus.cc:374] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:01.413229 10080 raft_consensus.cc:729] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1826fffa7ee54d4393a3a351c48cd8b8, State: Initialized, Role: FOLLOWER
I20260808 17:46:01.413163 10079 raft_consensus.cc:348] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.413261 10079 raft_consensus.cc:374] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:01.413287 10079 raft_consensus.cc:729] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b5658b628e47bfb8c7a8476c38ac60, State: Initialized, Role: FOLLOWER
I20260808 17:46:01.413280 10080 consensus_queue.cc:260] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.413338 10079 consensus_queue.cc:260] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.413360 10080 ts_tablet_manager.cc:1428] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:01.413419 10079 ts_tablet_manager.cc:1428] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 17:46:01.447829 10142 raft_consensus.cc:482] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:46:01.447928 10142 raft_consensus.cc:504] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.448140 10142 leader_election.cc:288] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969), 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561)
I20260808 17:46:01.448339 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d40abacb5540a982c0a3ae0e0e4cde" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" is_pre_election: true
I20260808 17:46:01.448324 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d40abacb5540a982c0a3ae0e0e4cde" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60" is_pre_election: true
I20260808 17:46:01.448457 9493 raft_consensus.cc:2454] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 0.
I20260808 17:46:01.448458 9964 raft_consensus.cc:2454] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 0.
I20260808 17:46:01.448642 9620 leader_election.cc:302] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1826fffa7ee54d4393a3a351c48cd8b8, 18b5658b628e47bfb8c7a8476c38ac60, f4dd378675864937bb745d4ade778eaf; no voters:
I20260808 17:46:01.448729 10142 raft_consensus.cc:2781] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:46:01.448770 10142 raft_consensus.cc:482] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:46:01.448840 10142 raft_consensus.cc:3037] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.449579 10142 raft_consensus.cc:504] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.449720 10142 leader_election.cc:288] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 election: Requested vote from peers 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969), 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561)
I20260808 17:46:01.449880 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d40abacb5540a982c0a3ae0e0e4cde" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60"
I20260808 17:46:01.449990 9493 raft_consensus.cc:3037] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.449954 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d40abacb5540a982c0a3ae0e0e4cde" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1826fffa7ee54d4393a3a351c48cd8b8"
I20260808 17:46:01.450023 9964 raft_consensus.cc:3037] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.450809 9493 raft_consensus.cc:2454] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 1.
I20260808 17:46:01.450875 9964 raft_consensus.cc:2454] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 1.
I20260808 17:46:01.451016 9620 leader_election.cc:302] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b5658b628e47bfb8c7a8476c38ac60, f4dd378675864937bb745d4ade778eaf; no voters:
I20260808 17:46:01.451109 10142 raft_consensus.cc:2781] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:01.451184 10142 raft_consensus.cc:686] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 1 LEADER]: Becoming Leader. State: Replica: f4dd378675864937bb745d4ade778eaf, State: Running, Role: LEADER
I20260808 17:46:01.451236 10142 consensus_queue.cc:237] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.451956 9227 catalog_manager.cc:5495] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf reported cstate change: term changed from 0 to 1, leader changed from <none> to f4dd378675864937bb745d4ade778eaf (127.29.62.194). New cstate: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } health_report { overall_health: UNKNOWN } } }
I20260808 17:46:01.461778 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.459507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46018 (local address 127.29.62.194:33183)
0808 17:46:01.459555 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.459557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.459575 (+ 18us) server_negotiation.cc:407] Connection header received
0808 17:46:01.459602 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.459603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.459613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.459666 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:46:01.459801 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.460285 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.461053 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.461266 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.461357 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.461504 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.461540 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.461611 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.461673 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.461696 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:46:01.462697 9964 raft_consensus.cc:1261] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Refusing update from remote peer f4dd378675864937bb745d4ade778eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:01.462795 9493 raft_consensus.cc:1261] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Refusing update from remote peer f4dd378675864937bb745d4ade778eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:01.463070 10142 consensus_queue.cc:1035] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:01.463174 10085 consensus_queue.cc:1035] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:01.466434 9227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:01.466544 9227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:01.467223 9227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: Sending DeleteTablet for 3 replicas of tablet e2d40abacb5540a982c0a3ae0e0e4cde
I20260808 17:46:01.467537 9463 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d40abacb5540a982c0a3ae0e0e4cde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:50054
I20260808 17:46:01.467563 9706 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d40abacb5540a982c0a3ae0e0e4cde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45686
I20260808 17:46:01.467636 10177 tablet_replica.cc:330] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: stopping tablet replica
I20260808 17:46:01.467650 10176 tablet_replica.cc:330] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: stopping tablet replica
I20260808 17:46:01.467690 10177 raft_consensus.cc:2229] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:46:01.467695 10176 raft_consensus.cc:2229] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:46:01.467736 10177 raft_consensus.cc:2258] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:01.467774 10176 raft_consensus.cc:2258] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:01.467931 9933 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d40abacb5540a982c0a3ae0e0e4cde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45114
I20260808 17:46:01.468024 10181 tablet_replica.cc:330] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: stopping tablet replica
I20260808 17:46:01.467986 9227 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 17:46:01.468083 10181 raft_consensus.cc:2229] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:46:01.468125 10181 raft_consensus.cc:2258] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 17:46:01.468145 9227 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:46:01.468446 10176 ts_tablet_manager.cc:1905] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:01.468583 10177 ts_tablet_manager.cc:1905] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:01.469707 9463 tablet_service.cc:1467] Processing CreateTablet for tablet 989ae6c7b63d448c9272b3eade381f45 (DEFAULT_TABLE table=test-table [id=b2e3d5101ab94999957ff6ab1dc20407]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:01.469832 9463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989ae6c7b63d448c9272b3eade381f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:01.470103 10177 ts_tablet_manager.cc:1918] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:46:01.470158 10177 log.cc:1196] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/e2d40abacb5540a982c0a3ae0e0e4cde
I20260808 17:46:01.470151 10176 ts_tablet_manager.cc:1918] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:46:01.470206 10176 log.cc:1196] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/e2d40abacb5540a982c0a3ae0e0e4cde
I20260808 17:46:01.470455 10177 ts_tablet_manager.cc:1939] T e2d40abacb5540a982c0a3ae0e0e4cde P 18b5658b628e47bfb8c7a8476c38ac60: Deleting consensus metadata
I20260808 17:46:01.470495 10176 ts_tablet_manager.cc:1939] T e2d40abacb5540a982c0a3ae0e0e4cde P f4dd378675864937bb745d4ade778eaf: Deleting consensus metadata
I20260808 17:46:01.470533 9933 tablet_service.cc:1467] Processing CreateTablet for tablet 989ae6c7b63d448c9272b3eade381f45 (DEFAULT_TABLE table=test-table [id=b2e3d5101ab94999957ff6ab1dc20407]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:01.470641 9933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989ae6c7b63d448c9272b3eade381f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:01.470942 10181 ts_tablet_manager.cc:1905] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:01.472133 10080 tablet_bootstrap.cc:492] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: Bootstrap starting.
I20260808 17:46:01.472203 9209 catalog_manager.cc:4841] TS 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969): tablet e2d40abacb5540a982c0a3ae0e0e4cde (table test-table [id=3bfcb76466e24d479d18c52871ee7306]) successfully deleted
I20260808 17:46:01.472378 10181 ts_tablet_manager.cc:1918] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:46:01.472432 10181 log.cc:1196] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/e2d40abacb5540a982c0a3ae0e0e4cde
I20260808 17:46:01.472467 10080 tablet_bootstrap.cc:654] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:01.472673 10181 ts_tablet_manager.cc:1939] T e2d40abacb5540a982c0a3ae0e0e4cde P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting consensus metadata
I20260808 17:46:01.473024 9208 catalog_manager.cc:4841] TS 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561): tablet e2d40abacb5540a982c0a3ae0e0e4cde (table test-table [id=3bfcb76466e24d479d18c52871ee7306]) successfully deleted
I20260808 17:46:01.473686 10079 tablet_bootstrap.cc:492] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60: Bootstrap starting.
I20260808 17:46:01.474074 9706 tablet_service.cc:1467] Processing CreateTablet for tablet 989ae6c7b63d448c9272b3eade381f45 (DEFAULT_TABLE table=test-table [id=b2e3d5101ab94999957ff6ab1dc20407]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:01.474195 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989ae6c7b63d448c9272b3eade381f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:01.474457 10079 tablet_bootstrap.cc:654] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:01.474496 9209 catalog_manager.cc:4841] TS f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183): tablet e2d40abacb5540a982c0a3ae0e0e4cde (table test-table [id=3bfcb76466e24d479d18c52871ee7306]) successfully deleted
I20260808 17:46:01.475067 10080 tablet_bootstrap.cc:492] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: No bootstrap required, opened a new log
I20260808 17:46:01.475129 10080 ts_tablet_manager.cc:1397] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 17:46:01.475256 10080 raft_consensus.cc:348] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.475348 10080 raft_consensus.cc:374] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:01.475369 10080 raft_consensus.cc:729] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1826fffa7ee54d4393a3a351c48cd8b8, State: Initialized, Role: FOLLOWER
I20260808 17:46:01.475425 10080 consensus_queue.cc:260] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.475520 10080 ts_tablet_manager.cc:1428] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:01.475733 10077 tablet_bootstrap.cc:492] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: Bootstrap starting.
I20260808 17:46:01.476181 10077 tablet_bootstrap.cc:654] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:01.476657 10079 tablet_bootstrap.cc:492] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60: No bootstrap required, opened a new log
I20260808 17:46:01.476713 10079 ts_tablet_manager.cc:1397] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 17:46:01.476840 10077 tablet_bootstrap.cc:492] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: No bootstrap required, opened a new log
I20260808 17:46:01.476892 10077 ts_tablet_manager.cc:1397] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:01.476872 10079 raft_consensus.cc:348] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.476974 10079 raft_consensus.cc:374] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:01.477001 10079 raft_consensus.cc:729] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b5658b628e47bfb8c7a8476c38ac60, State: Initialized, Role: FOLLOWER
I20260808 17:46:01.477022 10077 raft_consensus.cc:348] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.477047 10079 consensus_queue.cc:260] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.477090 10077 raft_consensus.cc:374] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:01.477109 10077 raft_consensus.cc:729] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4dd378675864937bb745d4ade778eaf, State: Initialized, Role: FOLLOWER
I20260808 17:46:01.477136 10079 ts_tablet_manager.cc:1428] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 17:46:01.477150 10077 consensus_queue.cc:260] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.477248 10077 ts_tablet_manager.cc:1428] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:01.516199 10087 raft_consensus.cc:482] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:46:01.516320 10087 raft_consensus.cc:504] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.516714 10087 leader_election.cc:288] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183), 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561)
I20260808 17:46:01.518431 10085 raft_consensus.cc:482] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:46:01.518512 10085 raft_consensus.cc:504] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.518651 10085 leader_election.cc:288] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969), 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561)
I20260808 17:46:01.518837 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989ae6c7b63d448c9272b3eade381f45" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60" is_pre_election: true
I20260808 17:46:01.518925 9493 raft_consensus.cc:2454] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 0.
I20260808 17:46:01.519068 9620 leader_election.cc:302] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b5658b628e47bfb8c7a8476c38ac60, f4dd378675864937bb745d4ade778eaf; no voters:
I20260808 17:46:01.519271 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989ae6c7b63d448c9272b3eade381f45" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" is_pre_election: true
I20260808 17:46:01.519368 9964 raft_consensus.cc:2454] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 0.
I20260808 17:46:01.519503 10085 raft_consensus.cc:2781] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:46:01.519549 10085 raft_consensus.cc:482] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:46:01.519574 10085 raft_consensus.cc:3037] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.520424 10085 raft_consensus.cc:504] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.520542 10085 leader_election.cc:288] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 election: Requested vote from peers 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969), 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561)
I20260808 17:46:01.520686 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.517010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:45157 (local address 127.29.62.195:37561)
0808 17:46:01.517060 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.517062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.517121 (+ 59us) server_negotiation.cc:407] Connection header received
0808 17:46:01.517246 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.517247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.517262 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.517297 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:01.517598 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.518069 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.519744 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.520290 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.520334 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.520344 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.520564 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.520604 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:46:01.520721 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989ae6c7b63d448c9272b3eade381f45" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18b5658b628e47bfb8c7a8476c38ac60"
I20260808 17:46:01.520785 9493 raft_consensus.cc:3037] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.520931 10222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.516915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37561 (local address 127.29.62.193:45157)
0808 17:46:01.517050 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.517062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:01.517157 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:01.517368 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:01.517369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:01.517370 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:01.517471 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.517484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.518175 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.518177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.519637 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.519640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.520400 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.520401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.520470 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.520486 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:46:01.520770 (+ 284us) client_negotiation.cc:240] Negotiation successful
0808 17:46:01.520826 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":93,"thread_start_us":65,"threads_started":1}
I20260808 17:46:01.521323 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989ae6c7b63d448c9272b3eade381f45" candidate_uuid: "f4dd378675864937bb745d4ade778eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1826fffa7ee54d4393a3a351c48cd8b8"
I20260808 17:46:01.521394 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.517097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:54941 (local address 127.29.62.194:33183)
0808 17:46:01.517138 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.517140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.517170 (+ 30us) server_negotiation.cc:407] Connection header received
0808 17:46:01.517247 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.517247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.517274 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.517338 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:01.517517 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.518238 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.520074 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.520618 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.520661 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.520668 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.521296 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.521338 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:46:01.521400 9964 raft_consensus.cc:3037] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:01.521486 9493 raft_consensus.cc:2454] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 1.
I20260808 17:46:01.521652 9620 leader_election.cc:302] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b5658b628e47bfb8c7a8476c38ac60, f4dd378675864937bb745d4ade778eaf; no voters:
I20260808 17:46:01.521734 10085 raft_consensus.cc:2781] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:01.521777 10085 raft_consensus.cc:686] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 1 LEADER]: Becoming Leader. State: Replica: f4dd378675864937bb745d4ade778eaf, State: Running, Role: LEADER
I20260808 17:46:01.521828 10085 consensus_queue.cc:237] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } }
I20260808 17:46:01.521878 10220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.516923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33183 (local address 127.29.62.193:54941)
0808 17:46:01.517110 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.517121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:01.517180 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:01.517356 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:01.517357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:01.517360 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:01.517436 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.517439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.518321 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.518321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.519971 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.519984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.520947 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.520948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.521200 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.521205 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:46:01.521643 (+ 438us) client_negotiation.cc:240] Negotiation successful
0808 17:46:01.521684 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":69,"threads_started":1}
I20260808 17:46:01.522336 9964 raft_consensus.cc:2454] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4dd378675864937bb745d4ade778eaf in term 1.
I20260808 17:46:01.522338 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989ae6c7b63d448c9272b3eade381f45" candidate_uuid: "18b5658b628e47bfb8c7a8476c38ac60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4dd378675864937bb745d4ade778eaf" is_pre_election: true
I20260808 17:46:01.522415 9229 catalog_manager.cc:5495] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf reported cstate change: term changed from 0 to 1, leader changed from <none> to f4dd378675864937bb745d4ade778eaf (127.29.62.194). New cstate: current_term: 1 leader_uuid: "f4dd378675864937bb745d4ade778eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4dd378675864937bb745d4ade778eaf" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 } health_report { overall_health: UNKNOWN } } }
I20260808 17:46:01.522599 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989ae6c7b63d448c9272b3eade381f45" candidate_uuid: "18b5658b628e47bfb8c7a8476c38ac60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" is_pre_election: true
I20260808 17:46:01.522675 9964 raft_consensus.cc:2379] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 18b5658b628e47bfb8c7a8476c38ac60 in current term 1: Already voted for candidate f4dd378675864937bb745d4ade778eaf in this term.
I20260808 17:46:01.522908 9369 leader_election.cc:302] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18b5658b628e47bfb8c7a8476c38ac60; no voters: 1826fffa7ee54d4393a3a351c48cd8b8, f4dd378675864937bb745d4ade778eaf
I20260808 17:46:01.523072 10087 raft_consensus.cc:2726] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 17:46:01.543098 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.540620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33183 (local address 127.29.62.194:40113)
0808 17:46:01.540682 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.540693 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:01.540741 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:01.540868 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:01.540869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:01.540870 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 17:46:01.540912 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.540913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.541743 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.541744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.542260 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:01.542262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.542714 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.542715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:01.542755 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.542761 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.542935 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.542937 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:46:01.542972 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 17:46:01.543019 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":2}
I20260808 17:46:01.543104 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.540680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:40113 (local address 127.29.62.194:33183)
0808 17:46:01.540717 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:01.540719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:01.540728 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:01.540786 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:01.540787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:01.540797 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:01.540860 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:46:01.540952 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.541674 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.542335 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:01.542644 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:01.542713 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:01.542853 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:46:01.542881 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:46:01.542932 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:01.542974 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 17:46:01.543003 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":18}
I20260808 17:46:01.545542 9493 raft_consensus.cc:1261] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Refusing update from remote peer f4dd378675864937bb745d4ade778eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:01.545568 9964 raft_consensus.cc:1261] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Refusing update from remote peer f4dd378675864937bb745d4ade778eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:01.545802 10085 consensus_queue.cc:1035] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "18b5658b628e47bfb8c7a8476c38ac60" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:01.545930 10085 consensus_queue.cc:1035] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1826fffa7ee54d4393a3a351c48cd8b8" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:01.556609 9227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:01.556741 9227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:01.557346 9227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d48cf38cc54799aca347f97d461918: Sending DeleteTablet for 3 replicas of tablet 989ae6c7b63d448c9272b3eade381f45
I20260808 17:46:01.557627 9933 tablet_service.cc:1514] Processing DeleteTablet for tablet 989ae6c7b63d448c9272b3eade381f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45114
I20260808 17:46:01.557669 9463 tablet_service.cc:1514] Processing DeleteTablet for tablet 989ae6c7b63d448c9272b3eade381f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:50054
I20260808 17:46:01.557778 10177 tablet_replica.cc:330] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60: stopping tablet replica
I20260808 17:46:01.557813 10181 tablet_replica.cc:330] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: stopping tablet replica
I20260808 17:46:01.557837 10177 raft_consensus.cc:2229] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:46:01.557878 10177 raft_consensus.cc:2258] T 989ae6c7b63d448c9272b3eade381f45 P 18b5658b628e47bfb8c7a8476c38ac60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:01.557884 10181 raft_consensus.cc:2229] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:46:01.557920 10181 raft_consensus.cc:2258] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:01.558066 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 9343
I20260808 17:46:01.564042 10181 ts_tablet_manager.cc:1905] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:01.566157 10181 ts_tablet_manager.cc:1918] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:46:01.566213 10181 log.cc:1196] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/989ae6c7b63d448c9272b3eade381f45
I20260808 17:46:01.566421 10181 ts_tablet_manager.cc:1939] T 989ae6c7b63d448c9272b3eade381f45 P 1826fffa7ee54d4393a3a351c48cd8b8: Deleting consensus metadata
I20260808 17:46:01.566737 9208 catalog_manager.cc:4841] TS 1826fffa7ee54d4393a3a351c48cd8b8 (127.29.62.195:37561): tablet 989ae6c7b63d448c9272b3eade381f45 (table test-table [id=b2e3d5101ab94999957ff6ab1dc20407]) successfully deleted
I20260808 17:46:01.567126 9704 tablet_service.cc:1514] Processing DeleteTablet for tablet 989ae6c7b63d448c9272b3eade381f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:01 UTC) from {username='oryx', principal='oryx/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45686
I20260808 17:46:01.567270 10176 tablet_replica.cc:330] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: stopping tablet replica
I20260808 17:46:01.567341 10176 raft_consensus.cc:2229] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:46:01.567408 10176 raft_consensus.cc:2258] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:01.567746 10176 ts_tablet_manager.cc:1905] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:01.569219 10176 ts_tablet_manager.cc:1918] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 17:46:01.569301 10176 log.cc:1196] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/989ae6c7b63d448c9272b3eade381f45
I20260808 17:46:01.569523 10176 ts_tablet_manager.cc:1939] T 989ae6c7b63d448c9272b3eade381f45 P f4dd378675864937bb745d4ade778eaf: Deleting consensus metadata
I20260808 17:46:01.569865 9209 catalog_manager.cc:4841] TS f4dd378675864937bb745d4ade778eaf (127.29.62.194:33183): tablet 989ae6c7b63d448c9272b3eade381f45 (table test-table [id=b2e3d5101ab94999957ff6ab1dc20407]) successfully deleted
W20260808 17:46:01.573794 9209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:35969 (error 108)
I20260808 17:46:01.574044 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 9580
I20260808 17:46:01.579782 10232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:01.579566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:35969 (local address 127.0.0.1:36596)
0808 17:46:01.579679 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:01.579738 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:35969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
W20260808 17:46:01.579936 9209 catalog_manager.cc:4584] TS 18b5658b628e47bfb8c7a8476c38ac60 (127.29.62.193:35969): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 989ae6c7b63d448c9272b3eade381f45: Network error: Client connection negotiation failed: client connection to 127.29.62.193:35969: connect: Connection refused (error 111)
I20260808 17:46:01.586573 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 9827
I20260808 17:46:01.599059 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 9181
2026-08-08T17:46:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3287 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:46:01 dist-test-slave-tk9z krb5kdc[10236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:46:01 dist-test-slave-tk9z krb5kdc[10236](info): set up 2 sockets
Aug 08 17:46:01 dist-test-slave-tk9z krb5kdc[10236](info): commencing operation
krb5kdc: starting...
W20260808 17:46:02.846804 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:46:02 dist-test-slave-tk9z krb5kdc[10236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:46:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:46:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:46:02.974153 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:33787
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:33787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:03.063687 10256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:03.063933 10256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:03.063972 10256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:03.065115 10256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:46:03.065189 10256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:03.065212 10256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:03.065233 10256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:46:03.065264 10256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:46:03.066808 10256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:33787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:33787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:03.067101 10256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:03.067338 10256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:03.072209 10262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:03.072196 10267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:03.072185 10263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:03.072466 10256 server_base.cc:1034] running on GCE node
I20260808 17:46:03.072727 10256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:03.073058 10256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:03.074225 10256 hybrid_clock.cc:648] HybridClock initialized: now 1786211163074207 us; error 47 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T17:46:03Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1477 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:46:03 dist-test-slave-tk9z krb5kdc[10275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:46:03 dist-test-slave-tk9z krb5kdc[10275](info): set up 2 sockets
Aug 08 17:46:03 dist-test-slave-tk9z krb5kdc[10275](info): commencing operation
krb5kdc: starting...
W20260808 17:46:04.326882 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:46:04 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:46:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:46:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:46:04.444810 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34741
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:34741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:04.526399 10300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:04.526605 10300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:04.526628 10300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:04.527770 10300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:46:04.527813 10300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:04.527827 10300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:04.527881 10300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:46:04.527915 10300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:46:04.529393 10300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:34741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:04.529649 10300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:04.529877 10300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:04.534643 10309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:04.534659 10312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:04.534648 10307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:04.534827 10300 server_base.cc:1034] running on GCE node
I20260808 17:46:04.535096 10300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:04.535398 10300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:04.536556 10300 hybrid_clock.cc:648] HybridClock initialized: now 1786211164536514 us; error 62 us; skew 500 ppm
Aug 08 17:46:04 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211164, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:04.544915 10300 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:46:04.545432 10300 webserver.cc:460] Webserver started at http://127.29.62.254:33609/ using document root <none> and password file <none>
I20260808 17:46:04.545672 10300 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:04.545742 10300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:04.545895 10300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:04.547065 10300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "4ccf74058cde4074bd32dafbd10fa2af"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "dd62c6d489d55d465d0f886aa150086e"
server_key_iv: "ce285c2900724e451d2cace46b1737dd"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.547447 10300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "4ccf74058cde4074bd32dafbd10fa2af"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "dd62c6d489d55d465d0f886aa150086e"
server_key_iv: "ce285c2900724e451d2cace46b1737dd"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.548800 10300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:46:04.549566 10323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:04.549738 10300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:04.549849 10300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "4ccf74058cde4074bd32dafbd10fa2af"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "dd62c6d489d55d465d0f886aa150086e"
server_key_iv: "ce285c2900724e451d2cace46b1737dd"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.549924 10300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:04.596855 10300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:04.599432 10300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:04.599611 10300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:04.603668 10300 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:34741
I20260808 17:46:04.603773 10413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:34741 every 8 connection(s)
I20260808 17:46:04.604125 10300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:46:04.605041 10416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:04.607375 10416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Bootstrap starting.
I20260808 17:46:04.607921 10416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:04.608161 10416 log.cc:824] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:04.608829 10416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: No bootstrap required, opened a new log
I20260808 17:46:04.610330 10416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } }
I20260808 17:46:04.610562 10416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:04.610535 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 10300
I20260808 17:46:04.610592 10416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ccf74058cde4074bd32dafbd10fa2af, State: Initialized, Role: FOLLOWER
I20260808 17:46:04.610677 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:46:04.610767 10416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } }
I20260808 17:46:04.610869 10416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:46:04.610915 10416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:46:04.610955 29947 external_mini_cluster.cc:1428] Setting key f748ecfea3ff776c7725a2408b7a2244
I20260808 17:46:04.610962 10416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:04.611640 10416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } }
I20260808 17:46:04.611771 10416 leader_election.cc:302] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ccf74058cde4074bd32dafbd10fa2af; no voters:
I20260808 17:46:04.611933 10416 leader_election.cc:288] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:46:04.611976 10423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:04.612116 10423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 1 LEADER]: Becoming Leader. State: Replica: 4ccf74058cde4074bd32dafbd10fa2af, State: Running, Role: LEADER
I20260808 17:46:04.612214 10416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:46:04.612272 10423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } }
I20260808 17:46:04.612701 10423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ccf74058cde4074bd32dafbd10fa2af. Latest consensus state: current_term: 1 leader_uuid: "4ccf74058cde4074bd32dafbd10fa2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } } }
I20260808 17:46:04.612739 10425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ccf74058cde4074bd32dafbd10fa2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } } }
I20260808 17:46:04.612931 10425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:04.613144 10423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:04.614017 10437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:46:04.614538 10437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:46:04.616134 10437 catalog_manager.cc:1347] Generated new cluster ID: dd968b0c1c7d4709a2fc3047c734662c
I20260808 17:46:04.616189 10437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:46:04 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:46:04.622680 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:04.611707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39168 (local address 127.29.62.254:34741)
0808 17:46:04.611957 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:04.611966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:04.612003 (+ 37us) server_negotiation.cc:407] Connection header received
0808 17:46:04.612129 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:04.612137 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:04.612198 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:04.613215 (+ 1017us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:04.613343 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:04.615354 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:04.616371 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:04.616800 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:04.616924 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:04.619340 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:04.619344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:04.619345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:04.619357 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:04.621087 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:04.621460 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:04.621463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:04.621463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:04.621568 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:04.621784 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:04.621785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:04.621786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:04.622037 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:04.622200 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:04.622302 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 17:46:04.622468 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":107,"threads_started":1}
I20260808 17:46:04.631042 10437 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:46:04.631578 10437 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:46:04.634266 10437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Generated new TSK 0
I20260808 17:46:04.634491 10437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:46:04.665760 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34741
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:04.793781 10457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:04.794067 10457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:04.794100 10457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:04.795948 10457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:04.796015 10457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:04.796056 10457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:46:04.798286 10457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:04.798650 10457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:04.799002 10457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:04.804615 10466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:04.804617 10469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:04.804615 10471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:04.805120 10457 server_base.cc:1034] running on GCE node
I20260808 17:46:04.805394 10457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:04.805686 10457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:04.806874 10457 hybrid_clock.cc:648] HybridClock initialized: now 1786211164806861 us; error 40 us; skew 500 ppm
Aug 08 17:46:04 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211164, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:04.810074 10457 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:46:04.810629 10457 webserver.cc:460] Webserver started at http://127.29.62.193:42821/ using document root <none> and password file <none>
I20260808 17:46:04.810902 10457 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:04.810950 10457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:04.811120 10457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:04.812235 10457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "fda8c703d1d54e6983bd894917dcc610"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "497f6665a2bb11055d81e8890dc6b84f"
server_key_iv: "d6227917d807f37b147c11e85760bccf"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.812697 10457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "fda8c703d1d54e6983bd894917dcc610"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "497f6665a2bb11055d81e8890dc6b84f"
server_key_iv: "d6227917d807f37b147c11e85760bccf"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.814191 10457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:46:04.815044 10482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:04.815219 10457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:04.815312 10457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "fda8c703d1d54e6983bd894917dcc610"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "497f6665a2bb11055d81e8890dc6b84f"
server_key_iv: "d6227917d807f37b147c11e85760bccf"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.815371 10457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:04.824007 10457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:04.826061 10457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:04.826176 10457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:04.826462 10457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:04.826746 10457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:04.826782 10457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:04.826805 10457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:04.826818 10457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:04.832641 10457 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:35235
I20260808 17:46:04.832801 10678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:35235 every 8 connection(s)
I20260808 17:46:04.833004 10457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:46:04 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211164, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:46:04.842182 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:04.833819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:56113 (local address 127.29.62.254:34741)
0808 17:46:04.833884 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:04.833886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:04.834049 (+ 163us) server_negotiation.cc:407] Connection header received
0808 17:46:04.834632 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:04.834634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:04.834649 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:04.834712 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:04.835773 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:04.836346 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:04.837322 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:04.837522 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:04.837581 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:04.839817 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:04.839818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:04.839818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:04.839825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:04.841063 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:04.841418 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:04.841418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:04.841419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:04.841471 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:04.841678 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:04.841679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:04.841679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:04.841775 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:04.841825 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:04.842052 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 17:46:04.842116 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:46:04.842372 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:04.833737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:56113)
0808 17:46:04.833916 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:04.833959 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:04.834541 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:04.834788 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:04.834792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:04.835367 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:04.835673 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:04.835680 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:04.836443 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:04.836444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:04.837232 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:04.837234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:04.837602 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:04.837603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:04.837704 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:04.838266 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:04.838275 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:04.839735 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:04.841133 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:04.841140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:04.841141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:04.841353 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:04.841529 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:04.841529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:04.841530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:04.841625 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:04.841828 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:04.841833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:04.841942 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 17:46:04.842024 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:46:04.842137 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260808 17:46:04.842972 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 10457
I20260808 17:46:04.843055 10679 heartbeater.cc:347] Connected to a master server at 127.29.62.254:34741
I20260808 17:46:04.843112 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:46:04.843161 10679 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:04.843331 29947 external_mini_cluster.cc:1428] Setting key 63554c4f88913b2f77abc2a327ec9265
I20260808 17:46:04.843364 10679 heartbeater.cc:511] Master 127.29.62.254:34741 requested a full tablet report, sending...
I20260808 17:46:04.843912 10350 ts_manager.cc:194] Registered new tserver with Master: fda8c703d1d54e6983bd894917dcc610 (127.29.62.193:35235)
I20260808 17:46:04.844549 10350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:56113
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:46:04.867308 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34741
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:04.960220 10692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:04.960427 10692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:04.960451 10692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:04.961621 10692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:04.961665 10692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:04.961721 10692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:46:04.963253 10692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:04.963500 10692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:04.963739 10692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:04.968667 10698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:04.968667 10703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:04.968659 10699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:04.968963 10692 server_base.cc:1034] running on GCE node
I20260808 17:46:04.969189 10692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:04.969451 10692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:04.970629 10692 hybrid_clock.cc:648] HybridClock initialized: now 1786211164970616 us; error 28 us; skew 500 ppm
Aug 08 17:46:04 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211164, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:04.973197 10692 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:46:04.973644 10692 webserver.cc:460] Webserver started at http://127.29.62.194:40771/ using document root <none> and password file <none>
I20260808 17:46:04.973873 10692 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:04.973941 10692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:04.974094 10692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:04.975215 10692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "221c5309e360485dbf210848ecda9bee"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "4a3982c7c0514c897e64b6645d48af1f"
server_key_iv: "d30bb548d285891050d07b6beaf55c25"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.975594 10692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "221c5309e360485dbf210848ecda9bee"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "4a3982c7c0514c897e64b6645d48af1f"
server_key_iv: "d30bb548d285891050d07b6beaf55c25"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.976951 10692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:46:04.977710 10711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:04.977890 10692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:46:04.978000 10692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "221c5309e360485dbf210848ecda9bee"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "4a3982c7c0514c897e64b6645d48af1f"
server_key_iv: "d30bb548d285891050d07b6beaf55c25"
server_key_version: "encryptionkey@0"
I20260808 17:46:04.978075 10692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:04.990751 10692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:04.992588 10692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:04.992753 10692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:04.992981 10692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:04.993316 10692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:04.993372 10692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:04.993414 10692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:04.993443 10692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:04.999364 10692 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:41631
I20260808 17:46:04.999411 10893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:41631 every 8 connection(s)
I20260808 17:46:04.999723 10692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:46:05.003437 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 10692
I20260808 17:46:05.003582 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:46:05.003786 29947 external_mini_cluster.cc:1428] Setting key 6013a8edea7b66a3544e9c4e77628535
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211164, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:46:05.010056 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.000438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:54803 (local address 127.29.62.254:34741)
0808 17:46:05.000498 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.000500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.000698 (+ 198us) server_negotiation.cc:407] Connection header received
0808 17:46:05.001239 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.001240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.001251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.001311 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.002304 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.003042 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.004254 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.004558 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.004635 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.006987 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:05.006989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:05.006989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:05.006996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:05.008629 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:05.009028 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:05.009030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:05.009030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:05.009103 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:05.009343 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:05.009344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:05.009344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:05.009509 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:05.009591 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:05.009841 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 17:46:05.009939 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:46:05.010128 10897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.000396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:54803)
0808 17:46:05.000578 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.000621 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.001140 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.001392 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.001395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.002024 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.002219 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.002223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.003150 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.003151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.004161 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.004165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.004653 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.004654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.004792 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.005327 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.005334 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.006880 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:05.008721 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:05.008727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:05.008729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:05.008952 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:05.009176 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:05.009176 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:05.009177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:05.009275 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:05.009578 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:05.009583 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:05.009702 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 17:46:05.009787 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:46:05.009912 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
I20260808 17:46:05.010718 10895 heartbeater.cc:347] Connected to a master server at 127.29.62.254:34741
I20260808 17:46:05.010835 10895 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:05.011029 10895 heartbeater.cc:511] Master 127.29.62.254:34741 requested a full tablet report, sending...
I20260808 17:46:05.011485 10350 ts_manager.cc:194] Registered new tserver with Master: 221c5309e360485dbf210848ecda9bee (127.29.62.194:41631)
I20260808 17:46:05.012025 10350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:54803
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:46:05.030995 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34741
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:05.115329 10906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:05.115532 10906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:05.115556 10906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:05.116658 10906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:05.116693 10906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:05.116719 10906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:46:05.118198 10906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:05.118464 10906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:05.118723 10906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:05.123660 10917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:05.123672 10915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:05.123871 10906 server_base.cc:1034] running on GCE node
W20260808 17:46:05.123674 10912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:05.124177 10906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:05.124572 10906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:05.125860 10906 hybrid_clock.cc:648] HybridClock initialized: now 1786211165125846 us; error 25 us; skew 500 ppm
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211165, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:05.128423 10906 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:46:05.128860 10906 webserver.cc:460] Webserver started at http://127.29.62.195:34845/ using document root <none> and password file <none>
I20260808 17:46:05.129101 10906 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:05.129171 10906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:05.129338 10906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:05.130539 10906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "be1aa671be6142bb9868ad6f8bb46807"
format_stamp: "Formatted at 2026-08-08 17:46:05 on dist-test-slave-tk9z"
server_key: "c3a6614b86fbaf1174f6b4a0eb4f053c"
server_key_iv: "7e8b6dbd830731c6323f2816a8378468"
server_key_version: "encryptionkey@0"
I20260808 17:46:05.130909 10906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "be1aa671be6142bb9868ad6f8bb46807"
format_stamp: "Formatted at 2026-08-08 17:46:05 on dist-test-slave-tk9z"
server_key: "c3a6614b86fbaf1174f6b4a0eb4f053c"
server_key_iv: "7e8b6dbd830731c6323f2816a8378468"
server_key_version: "encryptionkey@0"
I20260808 17:46:05.132234 10906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:05.133011 10927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.133191 10906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:05.133327 10906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "be1aa671be6142bb9868ad6f8bb46807"
format_stamp: "Formatted at 2026-08-08 17:46:05 on dist-test-slave-tk9z"
server_key: "c3a6614b86fbaf1174f6b4a0eb4f053c"
server_key_iv: "7e8b6dbd830731c6323f2816a8378468"
server_key_version: "encryptionkey@0"
I20260808 17:46:05.133406 10906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:05.154038 10906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:05.155995 10906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:05.156186 10906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:05.156458 10906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:05.156787 10906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:05.156847 10906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.156888 10906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:05.156919 10906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.162896 10906 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:41317
I20260808 17:46:05.162942 11092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:41317 every 8 connection(s)
I20260808 17:46:05.163302 10906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:46:05.166018 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 10906
I20260808 17:46:05.166131 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:46:05.166320 29947 external_mini_cluster.cc:1428] Setting key e98c4b61acd1853b5edc9e8ac1652f16
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211165, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:46:05.172561 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.164076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:45199 (local address 127.29.62.254:34741)
0808 17:46:05.164134 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.164136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.164336 (+ 200us) server_negotiation.cc:407] Connection header received
0808 17:46:05.164805 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.164806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.164815 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.164871 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.165860 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.166513 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.167672 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.167853 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.167907 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.169914 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:05.169916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:05.169916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:05.169923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:05.171390 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:05.171753 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:05.171754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:05.171754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:05.171808 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:05.172010 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:05.172011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:05.172011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:05.172108 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:05.172176 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:05.172426 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 17:46:05.172496 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:46:05.172748 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.164047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:45199)
0808 17:46:05.164202 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.164243 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.164724 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.164923 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.164926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.165591 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.165782 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.165787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.166589 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.166590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.167597 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.167600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.167928 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.167929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.168030 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.168434 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.168442 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.169827 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:05.171464 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:05.171470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:05.171471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:05.171688 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:05.171864 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:05.171865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:05.171865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:05.171959 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:05.172183 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:05.172188 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:05.172314 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 17:46:05.172399 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:46:05.172514 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260808 17:46:05.173243 11093 heartbeater.cc:347] Connected to a master server at 127.29.62.254:34741
I20260808 17:46:05.173367 11093 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:05.173554 11093 heartbeater.cc:511] Master 127.29.62.254:34741 requested a full tablet report, sending...
I20260808 17:46:05.173957 10350 ts_manager.cc:194] Registered new tserver with Master: be1aa671be6142bb9868ad6f8bb46807 (127.29.62.195:41317)
I20260808 17:46:05.174342 10350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:45199
I20260808 17:46:05.178108 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab.
Entry for principal oryx/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab.
WARNING: no policy specified for oryx/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab.
Entry for principal oryx/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab.
WARNING: no policy specified for oryx/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab.
Entry for principal oryx/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab.
I20260808 17:46:05.247198 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 10457
I20260808 17:46:05.257618 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 10692
I20260808 17:46:05.274945 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 10906
I20260808 17:46:05.287659 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 10300
I20260808 17:46:05.299683 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34741
--webserver_interface=127.29.62.254
--webserver_port=33609
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:34741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:05.381927 11103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:05.382167 11103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:05.382205 11103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:05.383368 11103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:46:05.383437 11103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:05.383463 11103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:05.383483 11103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:46:05.383515 11103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:46:05.385018 11103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:34741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=33609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:05.385303 11103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:05.385535 11103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:05.390482 11115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:05.390441 11121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:05.390501 11103 server_base.cc:1034] running on GCE node
W20260808 17:46:05.390436 11117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:05.390977 11103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:05.391317 11103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:05.392467 11103 hybrid_clock.cc:648] HybridClock initialized: now 1786211165392456 us; error 30 us; skew 500 ppm
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211165, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:05.395061 11103 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:46:05.395576 11103 webserver.cc:460] Webserver started at http://127.29.62.254:33609/ using document root <none> and password file <none>
I20260808 17:46:05.395835 11103 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:05.395934 11103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:05.397317 11103 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:05.397960 11135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.398149 11103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:05.398258 11103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "4ccf74058cde4074bd32dafbd10fa2af"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "dd62c6d489d55d465d0f886aa150086e"
server_key_iv: "ce285c2900724e451d2cace46b1737dd"
server_key_version: "encryptionkey@0"
I20260808 17:46:05.398689 11103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:05.409313 11103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:05.411325 11103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:05.411502 11103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:05.415545 11103 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:34741
I20260808 17:46:05.415581 11222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:34741 every 8 connection(s)
I20260808 17:46:05.415972 11103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:46:05.417435 11223 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:46:05.418042 11223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Bootstrap starting.
I20260808 17:46:05.424700 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 11103
I20260808 17:46:05.425078 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:35235
--local_ip_for_outbound_sockets=127.29.62.193
--tserver_master_addrs=127.29.62.254:34741
--webserver_port=42821
--webserver_interface=127.29.62.193
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.62.193
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:46:05.427302 11223 log.cc:824] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:05.429111 11223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:46:05.429361 11223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Bootstrap complete.
I20260808 17:46:05.431828 11223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } }
I20260808 17:46:05.432065 11223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ccf74058cde4074bd32dafbd10fa2af, State: Initialized, Role: FOLLOWER
I20260808 17:46:05.432241 11223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } }
I20260808 17:46:05.432323 11223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:46:05.432376 11223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:46:05.432426 11223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:46:05.433481 11223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } }
I20260808 17:46:05.433578 11223 leader_election.cc:302] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ccf74058cde4074bd32dafbd10fa2af; no voters:
I20260808 17:46:05.433804 11223 leader_election.cc:288] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:46:05.433840 11228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:46:05.433964 11228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [term 2 LEADER]: Becoming Leader. State: Replica: 4ccf74058cde4074bd32dafbd10fa2af, State: Running, Role: LEADER
I20260808 17:46:05.434101 11223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:46:05.434139 11228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } }
I20260808 17:46:05.434588 11230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4ccf74058cde4074bd32dafbd10fa2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } } }
I20260808 17:46:05.434687 11230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:05.434909 11231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ccf74058cde4074bd32dafbd10fa2af. Latest consensus state: current_term: 2 leader_uuid: "4ccf74058cde4074bd32dafbd10fa2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ccf74058cde4074bd32dafbd10fa2af" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34741 } } }
I20260808 17:46:05.435041 11231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:05.435179 11244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:46:05.435781 11244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:46:05.436030 11244 catalog_manager.cc:1259] Loaded cluster ID: dd968b0c1c7d4709a2fc3047c734662c
I20260808 17:46:05.436201 11244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:46:05.437180 11244 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:46:05.437376 11244 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ccf74058cde4074bd32dafbd10fa2af: Loaded TSK: 0
I20260808 17:46:05.437594 11244 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:46:05.520828 11225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:05.521045 11225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:05.521070 11225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:05.522235 11225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:05.522276 11225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:05.522303 11225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:46:05.523799 11225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.193.keytab
--principal=oryx/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:35235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=42821
--tserver_master_addrs=127.29.62.254:34741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:05.524055 11225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:05.524303 11225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:05.530122 11265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:05.530155 11267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:05.530144 11269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:05.530408 11225 server_base.cc:1034] running on GCE node
I20260808 17:46:05.530637 11225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:05.530881 11225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:05.532038 11225 hybrid_clock.cc:648] HybridClock initialized: now 1786211165532015 us; error 40 us; skew 500 ppm
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211165, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:05.534703 11225 init.cc:377] Logged in from keytab as oryx/127.29.62.193@KRBTEST.COM (short username oryx)
I20260808 17:46:05.535177 11225 webserver.cc:460] Webserver started at http://127.29.62.193:42821/ using document root <none> and password file <none>
I20260808 17:46:05.535418 11225 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:05.535511 11225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:05.536985 11225 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:46:05.537678 11281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.537935 11225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:05.538045 11225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "fda8c703d1d54e6983bd894917dcc610"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "497f6665a2bb11055d81e8890dc6b84f"
server_key_iv: "d6227917d807f37b147c11e85760bccf"
server_key_version: "encryptionkey@0"
I20260808 17:46:05.538555 11225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:05.562820 11225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:05.565317 11225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:05.565474 11225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:05.565760 11225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:05.566116 11225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:05.566174 11225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.566210 11225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:05.566237 11225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.572374 11225 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:35235
I20260808 17:46:05.572419 11462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:35235 every 8 connection(s)
I20260808 17:46:05.572759 11225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:05.581980 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 11225
I20260808 17:46:05.582099 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.573609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:49389)
0808 17:46:05.573777 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.573821 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.574301 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.574819 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.574822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.575395 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.575609 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.575614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.577247 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.577248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.578035 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.578037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.578866 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.578867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.578980 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.579549 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.579558 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.581863 (+ 2305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
W20260808 17:46:05.582162 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.573680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:49389 (local address 127.29.62.254:34741)
0808 17:46:05.573934 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.573944 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.573995 (+ 51us) server_negotiation.cc:407] Connection header received
0808 17:46:05.574543 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.574551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.574611 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.574768 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.575739 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.577139 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.578120 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.578774 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.578882 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.582009 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:49389: BlockingRecv error: recv got EOF from 127.29.62.193:49389 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":110,"threads_started":1}
I20260808 17:46:05.582340 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:41631
--local_ip_for_outbound_sockets=127.29.62.194
--tserver_master_addrs=127.29.62.254:34741
--webserver_port=40771
--webserver_interface=127.29.62.194
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.62.194
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:05.582475 11464 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:34741 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.62.254:34741: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:05.587072 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.582804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:44747)
0808 17:46:05.582902 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.582913 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.582960 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.583148 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.583149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.583286 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.583330 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.583332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.583991 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.583992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.584487 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.584489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.584860 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.584861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.584934 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.585377 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.585382 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.587024 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260808 17:46:05.587219 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.582900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:44747 (local address 127.29.62.254:34741)
0808 17:46:05.582937 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.582939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.582947 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:46:05.583054 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.583056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.583075 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.583147 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.583418 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.583893 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.584573 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.584769 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.584858 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.587131 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:44747: BlockingRecv error: recv got EOF from 127.29.62.193:44747 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:05.592121 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.587385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:47947)
0808 17:46:05.587429 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.587440 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.587485 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.587636 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.587637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.587739 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.587767 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.587768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.588683 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.588683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.589143 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.589144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.589649 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.589650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.589686 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.590158 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.590163 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.592076 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 17:46:05.592278 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.587434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:47947 (local address 127.29.62.254:34741)
0808 17:46:05.587471 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.587473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.587483 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:46:05.587552 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.587552 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.587564 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.587634 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.587844 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.588600 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.589225 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.589561 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.589637 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.592199 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:47947: BlockingRecv error: recv got EOF from 127.29.62.193:47947 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:46:05.592336 11464 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:46:05.669363 11468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:05.669569 11468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:05.669592 11468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:05.670831 11468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:05.670876 11468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:05.670902 11468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:46:05.672405 11468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.194.keytab
--principal=oryx/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:41631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=40771
--tserver_master_addrs=127.29.62.254:34741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:05.672665 11468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:05.672919 11468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:05.677953 11474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:05.677922 11480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:05.677911 11473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:05.678130 11468 server_base.cc:1034] running on GCE node
I20260808 17:46:05.678361 11468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:05.678635 11468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:05.679782 11468 hybrid_clock.cc:648] HybridClock initialized: now 1786211165679768 us; error 28 us; skew 500 ppm
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211165, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:05.682307 11468 init.cc:377] Logged in from keytab as oryx/127.29.62.194@KRBTEST.COM (short username oryx)
I20260808 17:46:05.682830 11468 webserver.cc:460] Webserver started at http://127.29.62.194:40771/ using document root <none> and password file <none>
I20260808 17:46:05.683091 11468 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:05.683178 11468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:05.684583 11468 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:05.685217 11491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.685407 11468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:46:05.685518 11468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "221c5309e360485dbf210848ecda9bee"
format_stamp: "Formatted at 2026-08-08 17:46:04 on dist-test-slave-tk9z"
server_key: "4a3982c7c0514c897e64b6645d48af1f"
server_key_iv: "d30bb548d285891050d07b6beaf55c25"
server_key_version: "encryptionkey@0"
I20260808 17:46:05.685886 11468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:05.722214 11468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:05.724282 11468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:05.724473 11468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:05.724752 11468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:05.725109 11468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:05.725178 11468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.725221 11468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:05.725250 11468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.731395 11468 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:41631
I20260808 17:46:05.731441 11672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:41631 every 8 connection(s)
I20260808 17:46:05.731873 11468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:05.739106 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 11468
I20260808 17:46:05.739440 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:41317
--local_ip_for_outbound_sockets=127.29.62.195
--tserver_master_addrs=127.29.62.254:34741
--webserver_port=34845
--webserver_interface=127.29.62.195
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.62.195
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:05.739998 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.732664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:52451 (local address 127.29.62.254:34741)
0808 17:46:05.732712 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.732716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.732963 (+ 247us) server_negotiation.cc:407] Connection header received
0808 17:46:05.733464 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.733465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.733479 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.733544 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.734650 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.735216 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.736313 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.736507 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.736558 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.739948 (+ 3390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:52451: BlockingRecv error: recv got EOF from 127.29.62.194:52451 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:46:05.740088 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.732642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:52451)
0808 17:46:05.732816 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.732861 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.733379 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.733627 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.733630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.734237 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.734556 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.734562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.735313 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.735314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.736224 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.736227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.736579 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.736580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.736690 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.737313 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.737322 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.739857 (+ 2535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
W20260808 17:46:05.740330 11679 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:34741 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.62.254:34741: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:05.745122 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.740622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:48103)
0808 17:46:05.740676 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.740685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.740740 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.740891 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.740891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.741028 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.741075 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.741076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.741723 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.741724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.742229 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.742231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.742661 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.742662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.742730 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.743165 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.743169 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.745037 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:46:05.745177 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.740678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:48103 (local address 127.29.62.254:34741)
0808 17:46:05.740738 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.740740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.740749 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:05.740801 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.740802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.740812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.740884 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.741152 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.741637 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.742315 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.742587 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.742663 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.745112 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:48103: BlockingRecv error: recv got EOF from 127.29.62.194:48103 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:05.750228 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.745460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:33077)
0808 17:46:05.745512 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.745522 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.745566 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.745720 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.745720 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.745824 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.745852 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.745853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.746801 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.746802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.747272 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.747273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.747784 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.747785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.747823 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.748275 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.748279 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.750132 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 17:46:05.750278 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.745519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:33077 (local address 127.29.62.254:34741)
0808 17:46:05.745590 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.745592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.745602 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:46:05.745629 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.745630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.745641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.745718 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.745927 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.746724 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.747355 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.747697 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.747771 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.750218 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:33077: BlockingRecv error: recv got EOF from 127.29.62.194:33077 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 17:46:05.750419 11679 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:46:05.824725 11684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:05.824931 11684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:05.824961 11684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:05.826110 11684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:05.826151 11684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:05.826177 11684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:46:05.827589 11684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/krb5kdc/oryx_127.29.62.195.keytab
--principal=oryx/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:41317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=34845
--tserver_master_addrs=127.29.62.254:34741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:05.827770 11684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:05.827935 11684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:05.832765 11700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:05.832784 11702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:05.832901 11698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:05.833132 11684 server_base.cc:1034] running on GCE node
I20260808 17:46:05.833289 11684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:05.833539 11684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:05.834758 11684 hybrid_clock.cc:648] HybridClock initialized: now 1786211165834740 us; error 34 us; skew 500 ppm
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211165, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:05.837158 11684 init.cc:377] Logged in from keytab as oryx/127.29.62.195@KRBTEST.COM (short username oryx)
I20260808 17:46:05.837718 11684 webserver.cc:460] Webserver started at http://127.29.62.195:34845/ using document root <none> and password file <none>
I20260808 17:46:05.837955 11684 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:05.838042 11684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:05.839421 11684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:05.840003 11714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.840178 11684 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:05.840276 11684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "be1aa671be6142bb9868ad6f8bb46807"
format_stamp: "Formatted at 2026-08-08 17:46:05 on dist-test-slave-tk9z"
server_key: "c3a6614b86fbaf1174f6b4a0eb4f053c"
server_key_iv: "7e8b6dbd830731c6323f2816a8378468"
server_key_version: "encryptionkey@0"
I20260808 17:46:05.840627 11684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:05.863535 11684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:05.865589 11684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:05.865779 11684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:05.866042 11684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:05.866407 11684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:05.866467 11684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.866508 11684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:05.866554 11684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:05.872200 11684 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:41317
I20260808 17:46:05.872332 11882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:41317 every 8 connection(s)
I20260808 17:46:05.872651 11684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:46:05.876029 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 11684
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 17:46:05.880667 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.873406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:33113 (local address 127.29.62.254:34741)
0808 17:46:05.873457 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.873459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.873719 (+ 260us) server_negotiation.cc:407] Connection header received
0808 17:46:05.874274 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.874276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.874290 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.874345 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.875353 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.875869 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.877251 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.877493 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.877559 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.880608 (+ 3049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:33113: BlockingRecv error: recv got EOF from 127.29.62.195:33113 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:46:05.880846 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.873403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:33113)
0808 17:46:05.873592 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.873628 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.874156 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.874469 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.874472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.875045 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.875237 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.875243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.875975 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.875976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.877167 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.877170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.877576 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.877577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.877700 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.878245 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.878254 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.880541 (+ 2287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
W20260808 17:46:05.881074 11884 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:34741 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.62.254:34741: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
I20260808 17:46:05.883757 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.876585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.29.62.254:34741)
0808 17:46:05.876715 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.876717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.876728 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:46:05.876784 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.876785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.876795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.876853 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.877431 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.877954 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.878869 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.879091 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.879175 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.880461 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:05.880465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:05.880465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:05.880477 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:05.882266 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:05.882694 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:05.882697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:05.882697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:05.882801 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:05.883016 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:05.883017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:05.883017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:05.883247 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:05.883402 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:05.883523 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 17:46:05.883641 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:05.885622 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.881296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:34311)
0808 17:46:05.881403 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.881413 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.881462 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.881593 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.881594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.881726 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.881768 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.881769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.882405 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.882406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.882899 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.882901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.883317 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.883317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.883383 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.883746 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.883750 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.885544 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260808 17:46:05.885672 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.881363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:34311 (local address 127.29.62.254:34741)
0808 17:46:05.881420 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.881421 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.881454 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:46:05.881530 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.881531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.881541 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.881591 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.881855 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.882297 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.882992 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.883242 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.883316 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.885610 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:34311: BlockingRecv error: recv got EOF from 127.29.62.195:34311 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:05 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:05.889658 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:05.885882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:52141)
0808 17:46:05.885934 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:05.885944 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:05.885980 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:05.886090 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:05.886091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:05.886213 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:05.886236 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.886237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.886875 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.886876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.887352 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:05.887354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.887696 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.887697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:05.887750 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.888061 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:05.888065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:05.889581 (+ 1516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 17:46:05.889729 11467 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:05.885941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:52141 (local address 127.29.62.254:34741)
0808 17:46:05.885991 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:05.885993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:05.886002 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:05.886024 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:05.886025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:05.886034 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:05.886077 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:05.886310 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.886783 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.887433 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:05.887622 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:05.887698 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:05.889646 (+ 1948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:52141: BlockingRecv error: recv got EOF from 127.29.62.195:52141 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 17:46:05.889922 11884 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 17:46:06 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:06 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:06.597564 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:06.592852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:40409)
0808 17:46:06.593022 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:06.593056 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:06.593131 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:06.593283 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:06.593284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:06.593450 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:06.593517 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:06.593519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.594153 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.594154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:06.594776 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:06.594778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.595131 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.595132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:06.595173 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:06.595612 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:06.595616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:06.597491 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
W20260808 17:46:06.597667 11891 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:06.592923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:40409 (local address 127.29.62.254:34741)
0808 17:46:06.593040 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:06.593042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:06.593138 (+ 96us) server_negotiation.cc:407] Connection header received
0808 17:46:06.593185 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:06.593186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:06.593199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:06.593263 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:06.593582 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.594084 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.594859 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.595062 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.595118 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:06.597591 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:40409: BlockingRecv error: recv got EOF from 127.29.62.193:40409 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
Aug 08 17:46:06 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:06 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:06.755904 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:06.750863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:44027)
0808 17:46:06.751160 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:06.751173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:06.751251 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:06.751414 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:06.751415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:06.751622 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:06.751674 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:06.751675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.752392 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.752394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:06.752963 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:06.752964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.753356 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.753357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:06.753419 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:06.753915 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:06.753919 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:06.755795 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":99,"threads_started":1}
W20260808 17:46:06.755962 11891 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:06.750958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:44027 (local address 127.29.62.254:34741)
0808 17:46:06.751150 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:06.751153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:06.751231 (+ 78us) server_negotiation.cc:407] Connection header received
0808 17:46:06.751321 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:06.751324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:06.751342 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:06.751407 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:06.751787 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.752299 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.753062 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.753261 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.753353 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:06.755896 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:44027: BlockingRecv error: recv got EOF from 127.29.62.194:44027 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Aug 08 17:46:06 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:06 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:06.895226 11893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:06.890342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:49509)
0808 17:46:06.890515 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:06.890526 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:06.890591 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:06.890735 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:06.890735 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:06.890926 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:06.890983 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:06.890985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.891618 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.891619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:06.892223 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:06.892224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.892542 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.892543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:06.892588 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:06.893041 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:06.893045 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:06.895111 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
W20260808 17:46:06.895268 11891 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:06.890441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:49509 (local address 127.29.62.254:34741)
0808 17:46:06.890503 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:06.890505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:06.890608 (+ 103us) server_negotiation.cc:407] Connection header received
0808 17:46:06.890640 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:06.890641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:06.890654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:06.890711 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:06.891052 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.891546 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.892295 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:06.892472 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:06.892523 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:06.895201 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:49509: BlockingRecv error: recv got EOF from 127.29.62.195:49509 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 17:46:07 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:07 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:07.603348 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:07.598135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:60973)
0808 17:46:07.598353 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:07.598367 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:07.598447 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:07.598627 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:07.598628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:07.598872 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:07.598942 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:07.598944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.599645 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.599646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:07.600197 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:07.600198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.600578 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.600579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:07.600642 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:07.601223 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:07.601228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:07.603247 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":139,"threads_started":1}
W20260808 17:46:07.603413 11897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:07.598270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:60973 (local address 127.29.62.254:34741)
0808 17:46:07.598447 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:07.598449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:07.598461 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:46:07.598521 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:07.598521 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:07.598533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:07.598616 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:07.599036 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.599548 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.600286 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.600499 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.600579 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:07.603328 (+ 2749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:60973: BlockingRecv error: recv got EOF from 127.29.62.193:60973 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":66,"threads_started":1}
Aug 08 17:46:07 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:07 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:07.761058 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:07.756440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:45799)
0808 17:46:07.756574 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:07.756585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:07.756655 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:07.756782 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:07.756783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:07.756946 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:07.757036 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:07.757038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.757670 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.757671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:07.758192 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:07.758193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.758631 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.758633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:07.758690 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:07.759152 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:07.759157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:07.760998 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
W20260808 17:46:07.761153 11897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:07.756489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:45799 (local address 127.29.62.254:34741)
0808 17:46:07.756558 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:07.756560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:07.756651 (+ 91us) server_negotiation.cc:407] Connection header received
0808 17:46:07.756703 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:07.756703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:07.756715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:07.756768 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:07.757099 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.757599 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.758304 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.758533 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.758599 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:07.761085 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:45799: BlockingRecv error: recv got EOF from 127.29.62.194:45799 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 17:46:07 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:07 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:07.900455 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:07.895752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:54815)
0808 17:46:07.895955 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:07.895969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:07.896062 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:07.896226 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:07.896227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:07.896414 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:07.896464 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:07.896466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.897137 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.897138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:07.897664 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:07.897666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.898040 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.898041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:07.898101 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:07.898598 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:07.898602 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:07.900395 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":118,"threads_started":1}
W20260808 17:46:07.900563 11897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:07.895890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:54815 (local address 127.29.62.254:34741)
0808 17:46:07.895959 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:07.895961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:07.896043 (+ 82us) server_negotiation.cc:407] Connection header received
0808 17:46:07.896119 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:07.896120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:07.896136 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:07.896204 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:07.896550 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.897050 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.897751 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:07.897946 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:07.898037 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:07.900492 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:54815: BlockingRecv error: recv got EOF from 127.29.62.195:54815 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 17:46:08 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:08 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:08.608574 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:08.603831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:41033)
0808 17:46:08.604009 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:08.604022 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:08.604092 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:08.604288 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:08.604289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:08.604460 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:08.604527 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:08.604529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.605165 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.605166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:08.605734 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:08.605736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.606080 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.606081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:08.606121 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:08.606622 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:08.606626 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:08.608513 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":67,"threads_started":1}
W20260808 17:46:08.608718 11904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:08.604029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:41033 (local address 127.29.62.254:34741)
0808 17:46:08.604146 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:08.604149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:08.604157 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:46:08.604211 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:08.604212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:08.604223 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:08.604281 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:08.604596 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.605098 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.605813 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.606011 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.606068 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:08.608604 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:41033: BlockingRecv error: recv got EOF from 127.29.62.193:41033 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
Aug 08 17:46:08 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:08 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:08.766533 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:08.761673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:37323)
0808 17:46:08.761825 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:08.761841 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:08.761905 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:08.762040 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:08.762041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:08.762210 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:08.762263 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:08.762264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.762961 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.762962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:08.763495 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:08.763497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.763815 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.763816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:08.763855 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:08.764291 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:08.764295 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:08.766430 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
W20260808 17:46:08.766599 11904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:08.761767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:37323 (local address 127.29.62.254:34741)
0808 17:46:08.761847 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:08.761849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:08.761889 (+ 40us) server_negotiation.cc:407] Connection header received
0808 17:46:08.761955 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:08.761956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:08.761967 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:08.762022 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:08.762331 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.762893 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.763570 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.763751 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.763802 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:08.766534 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:37323: BlockingRecv error: recv got EOF from 127.29.62.194:37323 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 17:46:08 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:08 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:08.906097 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:08.901006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:57055)
0808 17:46:08.901208 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:08.901218 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:08.901306 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:08.901448 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:08.901449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:08.901649 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:08.901700 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:08.901702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.902473 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.902475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:08.903075 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:08.903077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.903422 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.903423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:08.903469 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:08.903947 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:08.903952 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:08.905989 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":141,"threads_started":1}
W20260808 17:46:08.906139 11904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:08.901040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:57055 (local address 127.29.62.254:34741)
0808 17:46:08.901104 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:08.901106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:08.901287 (+ 181us) server_negotiation.cc:407] Connection header received
0808 17:46:08.901365 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:08.901366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:08.901382 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:08.901440 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:08.901789 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.902353 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.903158 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:08.903342 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:08.903395 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:08.906074 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:57055: BlockingRecv error: recv got EOF from 127.29.62.195:57055 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 17:46:09 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:09 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:09.617700 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:09.609050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:53591)
0808 17:46:09.609206 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:09.609217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:09.609271 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:09.609396 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:09.609396 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:09.609572 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:09.609626 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:09.609627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.610308 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.610309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:09.610965 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:09.610968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.611342 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.611343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:09.611387 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:09.615784 (+ 4397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:09.615789 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:09.617605 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
W20260808 17:46:09.617758 11909 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:09.609137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:53591 (local address 127.29.62.254:34741)
0808 17:46:09.609282 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:09.609284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:09.609292 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:46:09.609315 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:09.609315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:09.609325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:09.609384 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:09.609691 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.610206 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.611050 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.611248 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.611303 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:09.617684 (+ 6381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:53591: BlockingRecv error: recv got EOF from 127.29.62.193:53591 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
Aug 08 17:46:09 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:09 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:09.771864 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:09.767162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:50247)
0808 17:46:09.767330 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:09.767344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:09.767413 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:09.767542 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:09.767543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:09.767716 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:09.767769 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:09.767770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.768436 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.768437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:09.768991 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:09.768992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.769358 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.769360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:09.769410 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:09.769865 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:09.769870 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:09.771801 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
W20260808 17:46:09.772007 11909 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:09.767217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:50247 (local address 127.29.62.254:34741)
0808 17:46:09.767269 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:09.767271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:09.767449 (+ 178us) server_negotiation.cc:407] Connection header received
0808 17:46:09.767475 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:09.767476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:09.767489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:09.767538 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:09.767827 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.768327 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.769075 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.769266 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.769323 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:09.771923 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:50247: BlockingRecv error: recv got EOF from 127.29.62.194:50247 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 17:46:09 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:09 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:09.911835 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:09.906768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:40107)
0808 17:46:09.906927 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:09.906963 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:09.907033 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:09.907210 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:09.907211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:09.907424 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:09.907476 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:09.907478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.908417 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.908419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:09.908950 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:09.908951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.909330 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.909331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:09.909401 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:09.909843 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:09.909848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:09.911736 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
W20260808 17:46:09.911887 11909 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:09.906924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:40107 (local address 127.29.62.254:34741)
0808 17:46:09.906994 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:09.906996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:09.907010 (+ 14us) server_negotiation.cc:407] Connection header received
0808 17:46:09.907112 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:09.907113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:09.907129 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:09.907202 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:09.907567 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.908343 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.909039 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:09.909248 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:09.909331 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:09.911822 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:40107: BlockingRecv error: recv got EOF from 127.29.62.195:40107 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 17:46:10 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:10 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:10.622794 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:10.618212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:46313)
0808 17:46:10.618356 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:10.618389 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:10.618445 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:10.618620 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:10.618621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:10.618795 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:10.618848 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:10.618850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.619510 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.619511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:10.620021 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:10.620022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.620360 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.620361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:10.620400 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:10.620877 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:10.620881 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:10.622694 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
W20260808 17:46:10.622864 11917 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:10.618323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:46313 (local address 127.29.62.254:34741)
0808 17:46:10.618494 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:10.618496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:10.618504 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:46:10.618526 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:10.618527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:10.618538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:10.618601 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:10.618919 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.619440 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.620090 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.620276 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.620345 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:10.622779 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:46313: BlockingRecv error: recv got EOF from 127.29.62.193:46313 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
Aug 08 17:46:10 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:10 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:10.777182 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:10.772374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:48857)
0808 17:46:10.772537 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:10.772549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:10.772613 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:10.772783 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:10.772784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:10.772975 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:10.773026 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:10.773028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.773738 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.773740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:10.774275 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:10.774276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.774670 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.774671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:10.774734 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:10.775201 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:10.775206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:10.777124 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
W20260808 17:46:10.777287 11917 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:10.772544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:48857 (local address 127.29.62.254:34741)
0808 17:46:10.772604 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:10.772607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:10.772617 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:46:10.772673 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:10.772674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:10.772689 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:10.772762 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:10.773115 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.773644 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.774363 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.774590 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.774670 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:10.777222 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:48857: BlockingRecv error: recv got EOF from 127.29.62.194:48857 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 17:46:10 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:10 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:10.918203 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:10.912362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:33515)
0808 17:46:10.913041 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:10.913055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:10.913155 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:10.913364 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:10.913365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:10.913581 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:10.913637 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:10.913639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.914346 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.914347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:10.915157 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:10.915160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.915629 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.915630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:10.915679 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:10.916135 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:10.916140 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:10.918142 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":94,"threads_started":1}
W20260808 17:46:10.918282 11917 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:10.912432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:33515 (local address 127.29.62.254:34741)
0808 17:46:10.912504 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:10.912507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:10.913135 (+ 628us) server_negotiation.cc:407] Connection header received
0808 17:46:10.913251 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:10.913253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:10.913280 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:10.913355 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:10.913720 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.914270 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.915312 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:10.915559 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:10.915618 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:10.918233 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:33515: BlockingRecv error: recv got EOF from 127.29.62.195:33515 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 17:46:11 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:11 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:11.627954 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:11.623300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:53753)
0808 17:46:11.623473 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:11.623486 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:11.623541 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:11.623669 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:11.623670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:11.623848 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:11.623897 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:11.623899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.624582 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.624583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:11.625103 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:11.625104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.625451 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.625452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:11.625497 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:11.625949 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:11.625953 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:11.627895 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
W20260808 17:46:11.628116 11921 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:11.623396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:53753 (local address 127.29.62.254:34741)
0808 17:46:11.623529 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:11.623531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:11.623541 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:46:11.623591 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:11.623591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:11.623603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:11.623652 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:11.623982 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.624510 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.625181 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.625387 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.625441 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:11.628019 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:53753: BlockingRecv error: recv got EOF from 127.29.62.193:53753 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
Aug 08 17:46:11 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:11 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:11.782289 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:11.777753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:49085)
0808 17:46:11.777901 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:11.777910 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:11.777967 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:11.778119 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:11.778120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:11.778289 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:11.778341 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:11.778343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.779058 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.779058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:11.779590 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:11.779591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.779915 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.779915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:11.779960 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:11.780400 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:11.780421 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:11.782227 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
W20260808 17:46:11.782436 11921 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:11.777845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:49085 (local address 127.29.62.254:34741)
0808 17:46:11.777911 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:11.777913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:11.777951 (+ 38us) server_negotiation.cc:407] Connection header received
0808 17:46:11.778017 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:11.778018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:11.778030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:11.778092 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:11.778497 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.778988 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.779666 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.779846 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.779903 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:11.782332 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:49085: BlockingRecv error: recv got EOF from 127.29.62.194:49085 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 17:46:11 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:11 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:11.923522 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:11.918830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:49783)
0808 17:46:11.919087 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:11.919099 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:11.919162 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:11.919303 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:11.919304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:11.919494 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:11.919547 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:11.919549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.920171 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.920172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:11.920690 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:11.920692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.921027 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.921027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:11.921066 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:11.921507 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:11.921512 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:11.923424 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":74,"threads_started":1}
W20260808 17:46:11.923571 11921 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:11.918881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:49783 (local address 127.29.62.254:34741)
0808 17:46:11.918935 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:11.918937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:11.919167 (+ 230us) server_negotiation.cc:407] Connection header received
0808 17:46:11.919213 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:11.919214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:11.919226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:11.919286 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:11.919615 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.920103 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.920774 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:11.920959 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:11.921020 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:11.923504 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:49783: BlockingRecv error: recv got EOF from 127.29.62.195:49783 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 17:46:12 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:12 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:12.633114 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:12.628429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:44707)
0808 17:46:12.628612 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:12.628628 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:12.628683 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:12.628846 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:12.628847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:12.629031 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:12.629084 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:12.629086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.629721 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.629722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:12.630239 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:12.630240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.630605 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.630606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:12.630649 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:12.631157 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:12.631162 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:12.633015 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":113,"threads_started":1}
W20260808 17:46:12.633178 11927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:12.628548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:44707 (local address 127.29.62.254:34741)
0808 17:46:12.628708 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:12.628710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:12.628719 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:12.628751 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:12.628752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:12.628763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:12.628827 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:12.629155 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.629657 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.630326 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.630541 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.630596 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:12.633100 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:44707: BlockingRecv error: recv got EOF from 127.29.62.193:44707 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
Aug 08 17:46:12 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:12 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:12.787866 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:12.782886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:53077)
0808 17:46:12.783078 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:12.783090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:12.783148 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:12.783291 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:12.783292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:12.783480 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:12.783536 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:12.783537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.784180 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.784180 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:12.784715 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:12.784716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.785120 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.785121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:12.785186 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:12.785674 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:12.785678 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:12.787806 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":124,"threads_started":1}
W20260808 17:46:12.788029 11927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:12.782996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:53077 (local address 127.29.62.254:34741)
0808 17:46:12.783069 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:12.783071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:12.783132 (+ 61us) server_negotiation.cc:407] Connection header received
0808 17:46:12.783197 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:12.783197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:12.783210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:12.783269 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:12.783603 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.784108 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.784821 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.785028 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.785117 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:12.787939 (+ 2822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:53077: BlockingRecv error: recv got EOF from 127.29.62.194:53077 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 17:46:12 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:12 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:12.929179 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:12.924026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:42049)
0808 17:46:12.924205 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:12.924223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:12.924314 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:12.924489 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:12.924490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:12.924672 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:12.924725 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:12.924727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.925437 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.925438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:12.926228 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:12.926231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.926675 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.926676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:12.926740 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:12.927203 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:12.927208 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:12.929083 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
W20260808 17:46:12.929237 11927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:12.924114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:42049 (local address 127.29.62.254:34741)
0808 17:46:12.924190 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:12.924193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:12.924291 (+ 98us) server_negotiation.cc:407] Connection header received
0808 17:46:12.924392 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:12.924394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:12.924409 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:12.924481 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:12.924834 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.925341 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.926337 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:12.926588 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:12.926672 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:12.929168 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:42049: BlockingRecv error: recv got EOF from 127.29.62.195:42049 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 17:46:13 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:13 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:13.638523 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:13.633656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:34575)
0808 17:46:13.633854 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:13.633866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:13.633930 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:13.634069 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:13.634070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:13.634238 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:13.634289 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:13.634291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.635063 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.635064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:13.635630 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:13.635632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.636069 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.636070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:13.636133 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:13.636585 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:13.636590 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:13.638427 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
W20260808 17:46:13.638576 11937 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:13.633749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:34575 (local address 127.29.62.254:34741)
0808 17:46:13.633882 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:13.633884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:13.633919 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:46:13.633989 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:13.633989 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:13.634000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:13.634058 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:13.634349 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.634992 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.635741 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.635977 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.636065 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:13.638503 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:34575: BlockingRecv error: recv got EOF from 127.29.62.193:34575 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
Aug 08 17:46:13 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:13 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:13.793480 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:13.788338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:47889)
0808 17:46:13.788522 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:13.788537 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:13.788643 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:13.788827 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:13.788828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:13.789034 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:13.789086 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:13.789088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.789964 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.789965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:13.790539 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:13.790541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.790922 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.790923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:13.790994 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:13.791497 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:13.791507 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:13.793368 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
W20260808 17:46:13.793532 11937 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:13.788447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:47889 (local address 127.29.62.254:34741)
0808 17:46:13.788538 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:13.788540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:13.788622 (+ 82us) server_negotiation.cc:407] Connection header received
0808 17:46:13.788734 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:13.788735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:13.788748 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:13.788818 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:13.789177 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.789877 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.790632 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.790832 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.790919 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:13.793453 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:47889: BlockingRecv error: recv got EOF from 127.29.62.194:47889 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 17:46:13 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:13 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:13.934768 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:13.929734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:35355)
0808 17:46:13.929933 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:13.929948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:13.930062 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:13.930240 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:13.930241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:13.930476 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:13.930528 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:13.930531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.931228 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.931229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:13.931775 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:13.931777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.932151 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.932152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:13.932219 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:13.932721 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:13.932726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:13.934710 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
W20260808 17:46:13.934899 11937 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:13.929852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:35355 (local address 127.29.62.254:34741)
0808 17:46:13.929943 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:13.929945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:13.930042 (+ 97us) server_negotiation.cc:407] Connection header received
0808 17:46:13.930144 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:13.930145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:13.930163 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:13.930234 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:13.930628 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.931136 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.931865 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:13.932061 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:13.932151 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:13.934813 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:35355: BlockingRecv error: recv got EOF from 127.29.62.195:35355 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 17:46:14 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:14 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:14.646860 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:14.639010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:51593)
0808 17:46:14.639151 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:14.639161 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:14.639210 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:14.639340 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:14.639341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:14.639519 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:14.639568 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:14.639570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.640190 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.640191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:14.640700 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:14.640701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.641013 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.641014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:14.641052 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:14.644689 (+ 3637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:14.644695 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:14.646754 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
W20260808 17:46:14.646931 11943 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:14.639072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:51593 (local address 127.29.62.254:34741)
0808 17:46:14.639206 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:14.639208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:14.639217 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:14.639262 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:14.639262 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:14.639272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:14.639323 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:14.639633 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.640125 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.640766 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.640952 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.640999 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:14.646858 (+ 5859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:51593: BlockingRecv error: recv got EOF from 127.29.62.193:51593 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
Aug 08 17:46:14 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:14 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:14.798638 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:14.794010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:39461)
0808 17:46:14.794134 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:14.794152 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:14.794210 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:14.794325 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:14.794326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:14.794531 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:14.794583 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:14.794585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.795221 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.795221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:14.795744 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:14.795745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.796094 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.796095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:14.796137 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:14.796670 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:14.796676 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:14.798578 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
W20260808 17:46:14.798836 11943 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:14.794054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:39461 (local address 127.29.62.254:34741)
0808 17:46:14.794106 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:14.794108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:14.794194 (+ 86us) server_negotiation.cc:407] Connection header received
0808 17:46:14.794256 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:14.794257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:14.794270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:14.794322 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:14.794652 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.795149 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.795830 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.796023 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.796081 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:14.798767 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:39461: BlockingRecv error: recv got EOF from 127.29.62.194:39461 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 17:46:14 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:14 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:14.939819 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:14.935281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:32867)
0808 17:46:14.935442 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:14.935466 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:14.935528 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:14.935638 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:14.935639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:14.935844 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:14.935895 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:14.935897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.936524 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.936525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:14.937036 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:14.937037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.937344 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.937345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:14.937387 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:14.937901 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:14.937906 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:14.939749 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
W20260808 17:46:14.939956 11943 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:14.935340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:32867 (local address 127.29.62.254:34741)
0808 17:46:14.935385 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:14.935392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:14.935512 (+ 120us) server_negotiation.cc:407] Connection header received
0808 17:46:14.935571 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:14.935572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:14.935584 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:14.935654 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:14.935960 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.936447 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.937103 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:14.937279 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:14.937328 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:14.939842 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:32867: BlockingRecv error: recv got EOF from 127.29.62.195:32867 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 17:46:15 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:15 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:15.652472 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:15.647396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:40833)
0808 17:46:15.647551 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:15.647562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:15.647620 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:15.647791 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:15.647792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:15.647978 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:15.648031 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:15.648032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.648887 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.648888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:15.649565 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:15.649568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.649928 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.649929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:15.649977 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:15.650522 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:15.650527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:15.652406 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
W20260808 17:46:15.652618 11949 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:15.647515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:40833 (local address 127.29.62.254:34741)
0808 17:46:15.647663 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:15.647665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:15.647674 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:15.647700 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:15.647701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:15.647712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:15.647773 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:15.648098 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.648801 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.649643 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.649841 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.649898 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:15.652504 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:40833: BlockingRecv error: recv got EOF from 127.29.62.193:40833 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
Aug 08 17:46:15 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:15 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:15.804251 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:15.799124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:47393)
0808 17:46:15.799278 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:15.799289 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:15.799355 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:15.799508 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:15.799509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:15.799675 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:15.799728 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:15.799730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.800357 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.800358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:15.801015 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:15.801018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.801606 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.801607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:15.801653 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:15.802144 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:15.802150 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:15.804189 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
W20260808 17:46:15.804378 11949 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:15.799180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:47393 (local address 127.29.62.254:34741)
0808 17:46:15.799253 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:15.799255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:15.799353 (+ 98us) server_negotiation.cc:407] Connection header received
0808 17:46:15.799422 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:15.799423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:15.799434 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:15.799478 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:15.799790 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.800291 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.801149 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.801509 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.801594 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:15.804292 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:47393: BlockingRecv error: recv got EOF from 127.29.62.194:47393 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 17:46:15 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:15 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:15.945449 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:15.940311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:39631)
0808 17:46:15.940465 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:15.940486 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:15.940591 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:15.940787 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:15.940788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:15.941023 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:15.941090 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:15.941093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.941910 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.941911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:15.942507 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:15.942509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.942825 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.942826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:15.942866 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:15.943400 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:15.943405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:15.945367 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
W20260808 17:46:15.945556 11949 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:15.940461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:39631 (local address 127.29.62.254:34741)
0808 17:46:15.940526 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:15.940528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:15.940543 (+ 15us) server_negotiation.cc:407] Connection header received
0808 17:46:15.940681 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:15.940683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:15.940699 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:15.940762 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:15.941206 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.941825 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.942576 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:15.942758 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:15.942817 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:15.945487 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:39631: BlockingRecv error: recv got EOF from 127.29.62.195:39631 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 17:46:16 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:16 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:16.657846 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:16.652963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:60701)
0808 17:46:16.653130 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:16.653141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:16.653197 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:16.653399 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:16.653400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:16.653620 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:16.653686 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:16.653688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.654416 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.654417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:16.654957 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:16.654958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.655338 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.655339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:16.655427 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:16.655884 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:16.655888 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:16.657773 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
W20260808 17:46:16.658022 11957 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:16.653089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:60701 (local address 127.29.62.254:34741)
0808 17:46:16.653242 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:16.653244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:16.653278 (+ 34us) server_negotiation.cc:407] Connection header received
0808 17:46:16.653310 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:16.653311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:16.653322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:16.653384 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:16.653776 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.654302 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.655042 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.655249 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.655335 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:16.657909 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:60701: BlockingRecv error: recv got EOF from 127.29.62.193:60701 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
Aug 08 17:46:16 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:16 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:16.809603 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:16.804790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:54503)
0808 17:46:16.804945 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:16.804956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:16.805043 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:16.805223 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:16.805224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:16.805502 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:16.805595 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:16.805599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.806320 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.806321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:16.806887 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:16.806888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.807212 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.807213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:16.807253 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:16.807697 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:16.807702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:16.809530 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
W20260808 17:46:16.809700 11957 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:16.804841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:54503 (local address 127.29.62.254:34741)
0808 17:46:16.804929 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:16.804932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:16.805047 (+ 115us) server_negotiation.cc:407] Connection header received
0808 17:46:16.805117 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:16.805119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:16.805134 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:16.805204 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:16.805679 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.806247 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.806960 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.807147 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.807199 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:16.809630 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:54503: BlockingRecv error: recv got EOF from 127.29.62.194:54503 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 17:46:16 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:16 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:16.950896 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:16.945952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:34125)
0808 17:46:16.946100 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:16.946117 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:16.946183 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:16.946318 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:16.946319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:16.946618 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:16.946678 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:16.946681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.947320 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.947321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:16.947870 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:16.947871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.948198 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.948199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:16.948241 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:16.948672 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:16.948677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:16.950824 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":105,"threads_started":1}
W20260808 17:46:16.951017 11957 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:16.946018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:34125 (local address 127.29.62.254:34741)
0808 17:46:16.946104 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:16.946107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:16.946181 (+ 74us) server_negotiation.cc:407] Connection header received
0808 17:46:16.946229 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:16.946230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:16.946244 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:16.946295 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:16.946747 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.947251 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.947942 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:16.948133 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:16.948185 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:16.950939 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:34125: BlockingRecv error: recv got EOF from 127.29.62.195:34125 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 17:46:17 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:17 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:17.663697 11960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:17.658471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:50387)
0808 17:46:17.658635 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:17.658647 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:17.658708 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:17.658861 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:17.658862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:17.659042 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:17.659094 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:17.659096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.659742 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.659743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:17.660472 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:17.660475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.660996 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.660997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:17.661040 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:17.661651 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:17.661656 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:17.663636 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
W20260808 17:46:17.663846 11961 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:17.658529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:50387 (local address 127.29.62.254:34741)
0808 17:46:17.658679 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:17.658681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:17.658695 (+ 14us) server_negotiation.cc:407] Connection header received
0808 17:46:17.658777 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:17.658777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:17.658789 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:17.658843 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:17.659160 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.659674 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.660634 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.660922 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.660989 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:17.663737 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:50387: BlockingRecv error: recv got EOF from 127.29.62.193:50387 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
Aug 08 17:46:17 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:17 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:17.815125 11965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:17.810157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:58857)
0808 17:46:17.810352 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:17.810364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:17.810492 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:17.810667 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:17.810668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:17.810864 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:17.810914 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:17.810915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.811650 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.811652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:17.812395 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:17.812398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.812727 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.812728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:17.812770 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:17.813203 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:17.813207 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:17.815029 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":112,"threads_started":1}
W20260808 17:46:17.815177 11961 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:17.810248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:58857 (local address 127.29.62.254:34741)
0808 17:46:17.810337 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:17.810339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:17.810469 (+ 130us) server_negotiation.cc:407] Connection header received
0808 17:46:17.810564 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:17.810565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:17.810579 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:17.810658 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:17.811003 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.811547 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.812462 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.812656 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.812712 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:17.815111 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:58857: BlockingRecv error: recv got EOF from 127.29.62.194:58857 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 17:46:17 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:17 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:17.956494 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:17.951517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:54367)
0808 17:46:17.951711 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:17.951723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:17.951795 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:17.951939 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:17.951940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:17.952129 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:17.952182 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:17.952183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.952904 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.952905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:17.953628 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:17.953631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.953937 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.953938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:17.953989 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:17.954463 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:17.954468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:17.956393 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":116,"threads_started":1}
W20260808 17:46:17.956543 11961 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:17.951595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:54367 (local address 127.29.62.254:34741)
0808 17:46:17.951670 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:17.951672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:17.951792 (+ 120us) server_negotiation.cc:407] Connection header received
0808 17:46:17.951844 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:17.951845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:17.951860 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:17.951921 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:17.952246 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.952818 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.953683 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:17.953870 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:17.953936 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:17.956478 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:54367: BlockingRecv error: recv got EOF from 127.29.62.195:54367 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 17:46:18 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:18 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:18.668928 11968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:18.664214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:32997)
0808 17:46:18.664381 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:18.664393 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:18.664443 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:18.664589 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:18.664589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:18.664766 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:18.664816 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:18.664818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.665465 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.665465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:18.665992 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:18.665993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.666320 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.666321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:18.666360 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:18.666825 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:18.666829 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:18.668828 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
W20260808 17:46:18.669000 11970 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:18.664288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:32997 (local address 127.29.62.254:34741)
0808 17:46:18.664426 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:18.664427 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:18.664437 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:46:18.664503 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:18.664504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:18.664515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:18.664566 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:18.664880 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.665379 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.666066 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.666255 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.666304 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:18.668925 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:32997: BlockingRecv error: recv got EOF from 127.29.62.193:32997 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
Aug 08 17:46:18 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:18 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:18.820326 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:18.815643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:46325)
0808 17:46:18.815793 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:18.815803 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:18.815860 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:18.816034 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:18.816035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:18.816211 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:18.816262 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:18.816263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.816895 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.816896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:18.817466 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:18.817468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.817909 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.817910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:18.817976 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:18.818469 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:18.818474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:18.820254 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
W20260808 17:46:18.820456 11970 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:18.815806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:46325 (local address 127.29.62.254:34741)
0808 17:46:18.815872 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:18.815876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:18.815891 (+ 15us) server_negotiation.cc:407] Connection header received
0808 17:46:18.815951 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:18.815952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:18.815962 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:18.816013 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:18.816330 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.816827 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.817572 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.817828 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.817909 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:18.820354 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:46325: BlockingRecv error: recv got EOF from 127.29.62.194:46325 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 17:46:18 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:18 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:18.962081 11973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:18.957173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:59287)
0808 17:46:18.957358 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:18.957371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:18.957448 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:18.957600 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:18.957601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:18.957786 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:18.957840 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:18.957842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.958706 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.958707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:18.959260 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:18.959262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.959625 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.959626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:18.959668 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:18.960202 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:18.960207 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:18.962012 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":128,"threads_started":1}
W20260808 17:46:18.962235 11970 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:18.957259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:59287 (local address 127.29.62.254:34741)
0808 17:46:18.957331 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:18.957334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:18.957440 (+ 106us) server_negotiation.cc:407] Connection header received
0808 17:46:18.957512 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:18.957513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:18.957526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:18.957581 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:18.957938 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.958609 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.959346 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:18.959555 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:18.959616 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:18.962144 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:59287: BlockingRecv error: recv got EOF from 127.29.62.195:59287 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 17:46:19 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:19 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:19.678885 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:19.669440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:38743)
0808 17:46:19.669640 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:19.669654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:19.669723 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:19.669865 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:19.669866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:19.670044 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:19.670093 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:19.670094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.670875 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.670876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:19.671424 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:19.671426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.671773 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.671774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:19.671818 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:19.676957 (+ 5139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:19.676961 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:19.678813 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
W20260808 17:46:19.678951 11975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:19.669505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:38743 (local address 127.29.62.254:34741)
0808 17:46:19.669733 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:19.669735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:19.669744 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:46:19.669768 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:19.669768 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:19.669778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:19.669845 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:19.670167 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.670779 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.671504 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.671693 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.671744 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:19.678898 (+ 7154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:38743: BlockingRecv error: recv got EOF from 127.29.62.193:38743 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":111,"threads_started":1}
Aug 08 17:46:19 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:19 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:19.825644 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:19.820786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:47705)
0808 17:46:19.820950 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:19.820965 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:19.821076 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:19.821256 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:19.821257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:19.821447 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:19.821499 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:19.821501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.822168 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.822169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:19.822789 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:19.822792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.823187 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.823188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:19.823257 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:19.823759 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:19.823769 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:19.825545 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":58,"threads_started":1}
W20260808 17:46:19.825696 11975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:19.820970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:47705 (local address 127.29.62.254:34741)
0808 17:46:19.821036 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:19.821038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:19.821052 (+ 14us) server_negotiation.cc:407] Connection header received
0808 17:46:19.821159 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:19.821160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:19.821174 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:19.821248 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:19.821581 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.822081 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.822894 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.823094 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.823183 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:19.825630 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:47705: BlockingRecv error: recv got EOF from 127.29.62.194:47705 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 17:46:19 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:19 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.195@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:19.968046 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:19.962676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.195:33067)
0808 17:46:19.962824 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:19.962835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:19.962893 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:19.963038 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:19.963038 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:19.963217 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:19.963265 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:19.963266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.964142 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.964144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:19.965038 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:19.965041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.965430 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.965431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:19.965495 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:19.966078 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:19.966088 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:19.967986 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
W20260808 17:46:19.968209 11975 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:19.962782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:33067 (local address 127.29.62.254:34741)
0808 17:46:19.962848 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:19.962850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:19.962890 (+ 40us) server_negotiation.cc:407] Connection header received
0808 17:46:19.962954 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:19.962955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:19.962968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:19.963013 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:19.963338 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.964041 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.965128 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:19.965343 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:19.965429 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:19.968128 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.195:33067: BlockingRecv error: recv got EOF from 127.29.62.195:33067 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 17:46:20 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:20 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.193@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:20.684156 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:20.679513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.193:55169)
0808 17:46:20.679675 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:20.679686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:20.679744 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:20.679876 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:20.679877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:20.680047 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:20.680098 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:20.680099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:20.680724 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:20.680724 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:20.681242 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:20.681244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:20.681558 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:20.681559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:20.681599 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:20.682063 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:20.682068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:20.684095 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
W20260808 17:46:20.684334 11980 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:20.679623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:55169 (local address 127.29.62.254:34741)
0808 17:46:20.679753 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:20.679755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:20.679763 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:46:20.679787 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:20.679788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:20.679798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:20.679857 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:20.680166 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:20.680661 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:20.681313 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:20.681495 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:20.681543 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:20.684227 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.193:55169: BlockingRecv error: recv got EOF from 127.29.62.193:55169 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":55,"threads_started":1}
Aug 08 17:46:20 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 17:46:20 dist-test-slave-tk9z krb5kdc[10275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.62.194@KRBTEST.COM for oryx/127.29.62.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 17:46:20.831000 11981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:20.826295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34741 (local address 127.29.62.194:47641)
0808 17:46:20.826466 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:20.826478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:20.826545 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:20.826677 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:20.826678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:20.826839 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:20.826891 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:20.826892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:20.827612 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:20.827613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:20.828158 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:20.828160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:20.828532 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:20.828532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:20.828571 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:20.829064 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:20.829069 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:20.830913 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.62.254:34741: Server oryx/127.29.62.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":106,"threads_started":1}
W20260808 17:46:20.831133 11980 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:20.826310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:47641 (local address 127.29.62.254:34741)
0808 17:46:20.826411 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:20.826413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:20.826554 (+ 141us) server_negotiation.cc:407] Connection header received
0808 17:46:20.826589 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:20.826590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:20.826602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:20.826660 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:20.826979 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:20.827540 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:20.828241 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:20.828460 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:20.828523 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:20.831024 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.62.194:47641: BlockingRecv error: recv got EOF from 127.29.62.194:47641 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 17:46:20.880028 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 11225
I20260808 17:46:20.891398 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 11468
I20260808 17:46:20.904758 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 11684
I20260808 17:46:20.914826 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 11103
2026-08-08T17:46:20Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17837 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T17:46:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:46:20Z Disabled control of system clock
I20260808 17:46:20.950433 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:38049
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:38049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 17:46:21.033330 11988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:21.033538 11988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:21.033560 11988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:21.034736 11988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:46:21.034780 11988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:21.034792 11988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:21.034806 11988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:46:21.034818 11988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:46:21.036119 11988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:38049
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:38049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:21.036319 11988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:21.036641 11988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:21.042043 12002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.042124 11988 server_base.cc:1034] running on GCE node
W20260808 17:46:21.042060 11998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.042092 11999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.042665 11988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:21.042955 11988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:21.044133 11988 hybrid_clock.cc:648] HybridClock initialized: now 1786211181044109 us; error 38 us; skew 500 ppm
I20260808 17:46:21.045292 11988 webserver.cc:460] Webserver started at http://127.29.62.254:35445/ using document root <none> and password file <none>
I20260808 17:46:21.045534 11988 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:21.045604 11988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:21.045764 11988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:21.046924 11988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "df85f38ea5a94f65a4d4c7a9fb599292"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "1d97358aeefb77751331357acf5530bb"
server_key_iv: "43dacc620c57cedd498dd2d3f4cc0567"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.047299 11988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "df85f38ea5a94f65a4d4c7a9fb599292"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "1d97358aeefb77751331357acf5530bb"
server_key_iv: "43dacc620c57cedd498dd2d3f4cc0567"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.048713 11988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:46:21.049435 12010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.049605 11988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:21.049708 11988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "df85f38ea5a94f65a4d4c7a9fb599292"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "1d97358aeefb77751331357acf5530bb"
server_key_iv: "43dacc620c57cedd498dd2d3f4cc0567"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.049793 11988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:21.057960 11988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:21.058244 11988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:21.058456 11988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:21.062454 11988 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:38049
I20260808 17:46:21.062485 12093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:38049 every 8 connection(s)
I20260808 17:46:21.062826 11988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:46:21.063493 12094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:21.064776 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 11988
I20260808 17:46:21.064862 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:46:21.065086 29947 external_mini_cluster.cc:1428] Setting key 37bd1fa0c4d15d5f391b1f50e57f1a91
I20260808 17:46:21.066862 12094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292: Bootstrap starting.
I20260808 17:46:21.067605 12094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:21.067934 12094 log.cc:824] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:21.068732 12094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292: No bootstrap required, opened a new log
I20260808 17:46:21.071159 12094 raft_consensus.cc:348] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df85f38ea5a94f65a4d4c7a9fb599292" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38049 } }
I20260808 17:46:21.071321 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.065642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.29.62.254:38049)
0808 17:46:21.065859 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:21.065868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:21.065901 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:46:21.066543 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:21.066552 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:21.066632 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:21.066785 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:21.066949 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.068344 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.069389 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.069800 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.069895 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:21.070653 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:21.070657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:21.070657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:46:21.070814 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:21.070845 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:46:21.070855 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:21.070907 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:21.070997 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 17:46:21.071083 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260808 17:46:21.071347 12094 raft_consensus.cc:374] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:21.071393 12094 raft_consensus.cc:729] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df85f38ea5a94f65a4d4c7a9fb599292, State: Initialized, Role: FOLLOWER
I20260808 17:46:21.071539 12094 consensus_queue.cc:260] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df85f38ea5a94f65a4d4c7a9fb599292" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38049 } }
I20260808 17:46:21.071636 12094 raft_consensus.cc:388] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:46:21.071682 12094 raft_consensus.cc:482] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:46:21.071730 12094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:21.072335 12094 raft_consensus.cc:504] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df85f38ea5a94f65a4d4c7a9fb599292" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38049 } }
I20260808 17:46:21.072428 12094 leader_election.cc:302] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df85f38ea5a94f65a4d4c7a9fb599292; no voters:
I20260808 17:46:21.072559 12094 leader_election.cc:288] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:46:21.072670 12103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:21.072800 12094 sys_catalog.cc:564] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:46:21.072863 12103 raft_consensus.cc:686] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [term 1 LEADER]: Becoming Leader. State: Replica: df85f38ea5a94f65a4d4c7a9fb599292, State: Running, Role: LEADER
I20260808 17:46:21.072978 12103 consensus_queue.cc:237] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df85f38ea5a94f65a4d4c7a9fb599292" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38049 } }
I20260808 17:46:21.073392 12105 sys_catalog.cc:455] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df85f38ea5a94f65a4d4c7a9fb599292. Latest consensus state: current_term: 1 leader_uuid: "df85f38ea5a94f65a4d4c7a9fb599292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df85f38ea5a94f65a4d4c7a9fb599292" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38049 } } }
I20260808 17:46:21.073478 12105 sys_catalog.cc:458] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:21.073714 12113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:46:21.073704 12104 sys_catalog.cc:455] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df85f38ea5a94f65a4d4c7a9fb599292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df85f38ea5a94f65a4d4c7a9fb599292" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38049 } } }
I20260808 17:46:21.074088 12104 sys_catalog.cc:458] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292 [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:21.074360 12113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:46:21.076038 12113 catalog_manager.cc:1347] Generated new cluster ID: 6f21e609ed464ff1b71884666bb9d532
I20260808 17:46:21.076107 12113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:46:21.092549 12113 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:46:21.093138 12113 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:46:21.095523 12113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df85f38ea5a94f65a4d4c7a9fb599292: Generated new TSK 0
I20260808 17:46:21.095712 12113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:46:21.097651 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38049
--builtin_ntp_servers=127.29.62.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 17:46:21.185170 12129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:21.185382 12129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:21.185405 12129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:21.186633 12129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:21.186676 12129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:21.186700 12129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:46:21.188040 12129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:21.188313 12129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:21.188625 12129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:21.193671 12141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.193691 12136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.193897 12129 server_base.cc:1034] running on GCE node
W20260808 17:46:21.194022 12138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.194247 12129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:21.194520 12129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:21.195690 12129 hybrid_clock.cc:648] HybridClock initialized: now 1786211181195673 us; error 34 us; skew 500 ppm
I20260808 17:46:21.196970 12129 webserver.cc:460] Webserver started at http://127.29.62.193:43431/ using document root <none> and password file <none>
I20260808 17:46:21.197225 12129 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:21.197293 12129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:21.197443 12129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:21.198575 12129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "4032465767d14f819f99a04139b25258"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "c30528e00b1ab8978f13398a762a31fd"
server_key_iv: "8929ae321bdcd7ae6ae7e99ab8418b71"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.198941 12129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "4032465767d14f819f99a04139b25258"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "c30528e00b1ab8978f13398a762a31fd"
server_key_iv: "8929ae321bdcd7ae6ae7e99ab8418b71"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.200258 12129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:21.200984 12152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.201153 12129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:21.201257 12129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "4032465767d14f819f99a04139b25258"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "c30528e00b1ab8978f13398a762a31fd"
server_key_iv: "8929ae321bdcd7ae6ae7e99ab8418b71"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.201336 12129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:21.214087 12129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:21.214354 12129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:21.214578 12129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:21.214818 12129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:21.215133 12129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:21.215188 12129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.215222 12129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:21.215250 12129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.221203 12129 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:33443
I20260808 17:46:21.221253 12344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:33443 every 8 connection(s)
I20260808 17:46:21.221558 12129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:46:21.223630 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 12129
I20260808 17:46:21.223737 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:46:21.223930 29947 external_mini_cluster.cc:1428] Setting key e92f02ca213092bda53913a05c001bd7
I20260808 17:46:21.224848 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38049
--builtin_ntp_servers=127.29.62.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 17:46:21.227026 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.222203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:37991 (local address 127.29.62.254:38049)
0808 17:46:21.222270 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:21.222272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:21.222540 (+ 268us) server_negotiation.cc:407] Connection header received
0808 17:46:21.223016 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:21.223017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:21.223027 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:21.223085 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:21.223743 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.224400 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.225537 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.225752 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.225826 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:21.226757 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:21.226758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:21.226758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:46:21.226763 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:21.226778 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:46:21.226782 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:21.226829 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:21.226947 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 17:46:21.226968 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:46:21.227275 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.222175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38049 (local address 127.29.62.193:37991)
0808 17:46:21.222405 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:21.222444 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:21.222925 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:21.223146 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:21.223150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:21.223158 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:21.223643 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:21.223647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.224495 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.224496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:21.225446 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:21.225448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.225847 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.225848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:21.225985 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:21.226594 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:46:21.226594 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:21.226634 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:46:21.226662 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:21.226848 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:21.226855 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:21.226858 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:46:21.226929 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:46:21.227036 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260808 17:46:21.227838 12346 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38049
I20260808 17:46:21.227927 12346 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:21.228114 12346 heartbeater.cc:511] Master 127.29.62.254:38049 requested a full tablet report, sending...
I20260808 17:46:21.228591 12034 ts_manager.cc:194] Registered new tserver with Master: 4032465767d14f819f99a04139b25258 (127.29.62.193:33443)
I20260808 17:46:21.229142 12034 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.193:37991
W20260808 17:46:21.310163 12352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:21.310451 12352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:21.310500 12352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:21.311661 12352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:21.311726 12352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:21.311770 12352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:46:21.313112 12352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:21.313386 12352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:21.313694 12352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:21.318713 12358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.318717 12364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.318764 12360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.319028 12352 server_base.cc:1034] running on GCE node
I20260808 17:46:21.319240 12352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:21.319442 12352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:21.320605 12352 hybrid_clock.cc:648] HybridClock initialized: now 1786211181320593 us; error 27 us; skew 500 ppm
I20260808 17:46:21.321861 12352 webserver.cc:460] Webserver started at http://127.29.62.194:41969/ using document root <none> and password file <none>
I20260808 17:46:21.322134 12352 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:21.322204 12352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:21.322412 12352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:21.323534 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "878eb2ef76714c9e86a6f09a0bbce7b9"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "ebff8b8fd6b97da35ddeed67618cf2d0"
server_key_iv: "019ab3ca90342b5ce92bdab98ad4221f"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.323920 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "878eb2ef76714c9e86a6f09a0bbce7b9"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "ebff8b8fd6b97da35ddeed67618cf2d0"
server_key_iv: "019ab3ca90342b5ce92bdab98ad4221f"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.325307 12352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:46:21.326054 12370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.326231 12352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:21.326324 12352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "878eb2ef76714c9e86a6f09a0bbce7b9"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "ebff8b8fd6b97da35ddeed67618cf2d0"
server_key_iv: "019ab3ca90342b5ce92bdab98ad4221f"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.326434 12352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:21.338116 12352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:21.338459 12352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:21.338617 12352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:21.338851 12352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:21.339193 12352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:21.339251 12352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.339293 12352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:21.339320 12352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.345280 12352 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:41945
I20260808 17:46:21.345330 12556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:41945 every 8 connection(s)
I20260808 17:46:21.345674 12352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:46:21.350790 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.346397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:45783 (local address 127.29.62.254:38049)
0808 17:46:21.346448 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:21.346450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:21.346731 (+ 281us) server_negotiation.cc:407] Connection header received
0808 17:46:21.347228 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:21.347230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:21.347240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:21.347302 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:21.348004 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.348498 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.349421 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.349611 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.349663 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:21.350544 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:21.350546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:21.350546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:46:21.350551 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:21.350565 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:46:21.350569 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:21.350619 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:21.350708 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 17:46:21.350728 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:46:21.351012 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.346327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38049 (local address 127.29.62.194:45783)
0808 17:46:21.346549 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:21.346590 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:21.347124 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:21.347386 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:21.347390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:21.347398 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:21.347921 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:21.347925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.348579 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.348580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:21.349343 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:21.349345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.349686 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.349687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:21.349797 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:21.350393 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:46:21.350394 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:21.350440 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:46:21.350462 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:21.350633 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:21.350638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:21.350640 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:46:21.350694 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:46:21.350771 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":123,"threads_started":1}
I20260808 17:46:21.351141 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 12352
I20260808 17:46:21.351241 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:46:21.351467 29947 external_mini_cluster.cc:1428] Setting key c1d5a1a5fc93578977f4c74d4ba6d8fa
I20260808 17:46:21.351599 12557 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38049
I20260808 17:46:21.351703 12557 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:21.351907 12557 heartbeater.cc:511] Master 127.29.62.254:38049 requested a full tablet report, sending...
I20260808 17:46:21.352311 12034 ts_manager.cc:194] Registered new tserver with Master: 878eb2ef76714c9e86a6f09a0bbce7b9 (127.29.62.194:41945)
I20260808 17:46:21.352535 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38049
--builtin_ntp_servers=127.29.62.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 17:46:21.352795 12034 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.194:45783
W20260808 17:46:21.435822 12564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:21.436038 12564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:21.436058 12564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:21.437209 12564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:21.437250 12564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:21.437300 12564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:46:21.438757 12564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:21.438989 12564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:21.439287 12564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:21.444370 12580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.444530 12564 server_base.cc:1034] running on GCE node
W20260808 17:46:21.444386 12576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.444375 12577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.444854 12564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:21.445137 12564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:21.446295 12564 hybrid_clock.cc:648] HybridClock initialized: now 1786211181446290 us; error 35 us; skew 500 ppm
I20260808 17:46:21.447650 12564 webserver.cc:460] Webserver started at http://127.29.62.195:44687/ using document root <none> and password file <none>
I20260808 17:46:21.447903 12564 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:21.447979 12564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:21.448123 12564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:21.449167 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "0aae37aaeb0544eb9d94d324a0e8fb45"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "5bda72006ecf676dadb11f5e3a7eebe2"
server_key_iv: "8573c61fbb30bcbd016d8501b1c0b4dc"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.449538 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "0aae37aaeb0544eb9d94d324a0e8fb45"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "5bda72006ecf676dadb11f5e3a7eebe2"
server_key_iv: "8573c61fbb30bcbd016d8501b1c0b4dc"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.450990 12564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:46:21.451783 12593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.452020 12564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:21.452128 12564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "0aae37aaeb0544eb9d94d324a0e8fb45"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "5bda72006ecf676dadb11f5e3a7eebe2"
server_key_iv: "8573c61fbb30bcbd016d8501b1c0b4dc"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.452207 12564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:21.462837 12564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:21.463122 12564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:21.463268 12564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:21.463492 12564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:21.463835 12564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:21.463889 12564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.463932 12564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:21.463959 12564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.469801 12564 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:39647
I20260808 17:46:21.469911 12777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:39647 every 8 connection(s)
I20260808 17:46:21.470227 12564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:46:21.475608 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.471058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:48553 (local address 127.29.62.254:38049)
0808 17:46:21.471124 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:21.471126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:21.471346 (+ 220us) server_negotiation.cc:407] Connection header received
0808 17:46:21.471825 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:21.471826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:21.471835 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:21.471896 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:21.472605 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.473193 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.474304 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.474541 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.474598 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:21.475377 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:21.475378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:21.475379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:46:21.475383 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:21.475398 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:46:21.475401 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:21.475445 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:21.475527 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 17:46:21.475548 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:46:21.475805 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.471012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38049 (local address 127.29.62.195:48553)
0808 17:46:21.471211 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:21.471252 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:21.471737 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:21.471970 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:21.471974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:21.471982 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:21.472521 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:21.472527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.473335 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.473337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:21.474231 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:21.474234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:21.474640 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:21.474641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:21.474754 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:21.475249 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:46:21.475249 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:21.475277 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:46:21.475299 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:21.475455 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:21.475459 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:21.475462 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:46:21.475514 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:46:21.475593 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260808 17:46:21.476292 12778 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38049
I20260808 17:46:21.476382 12778 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:21.476531 12778 heartbeater.cc:511] Master 127.29.62.254:38049 requested a full tablet report, sending...
I20260808 17:46:21.476907 12034 ts_manager.cc:194] Registered new tserver with Master: 0aae37aaeb0544eb9d94d324a0e8fb45 (127.29.62.195:39647)
I20260808 17:46:21.477308 12034 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.195:48553
I20260808 17:46:21.478089 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 12564
I20260808 17:46:21.478173 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:46:21.478327 29947 external_mini_cluster.cc:1428] Setting key 71f0582a44e54d47879b35741054c1c8
I20260808 17:46:21.479130 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 17:46:21.480935 12099 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:21.480452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.29.62.254:38049)
0808 17:46:21.480498 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:21.480499 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:21.480638 (+ 139us) server_negotiation.cc:407] Connection header received
0808 17:46:21.480677 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:21.480677 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:21.480686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:21.480733 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:21.480850 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53500: BlockingRecv error: recv got EOF from 127.0.0.1:53500 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:46:21.481057 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 12129
I20260808 17:46:21.495222 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 12352
I20260808 17:46:21.517449 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 12564
I20260808 17:46:21.528328 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 11988
2026-08-08T17:46:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (608 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T17:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:46:21Z Disabled control of system clock
I20260808 17:46:21.558709 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:46807
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:46807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 17:46:21.642313 12794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:21.642647 12794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:21.642685 12794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:21.643841 12794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:46:21.643910 12794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:21.643934 12794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:21.643954 12794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:46:21.643981 12794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:46:21.645368 12794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:46807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:46807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:21.645674 12794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:21.645990 12794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:21.650915 12808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.650890 12805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.650918 12803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.651217 12794 server_base.cc:1034] running on GCE node
I20260808 17:46:21.651484 12794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:21.651829 12794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:21.653023 12794 hybrid_clock.cc:648] HybridClock initialized: now 1786211181653019 us; error 55 us; skew 500 ppm
I20260808 17:46:21.654294 12794 webserver.cc:460] Webserver started at http://127.29.62.254:40749/ using document root <none> and password file <none>
I20260808 17:46:21.654575 12794 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:21.654649 12794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:21.654790 12794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:21.655970 12794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "573fc994e0fb429fa4b1cce2fc2b03ea"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "dce01953fa602c1e97fe33ddd5220005"
server_key_iv: "7c95a078d401c2796dfbae0479637184"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.656376 12794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "573fc994e0fb429fa4b1cce2fc2b03ea"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "dce01953fa602c1e97fe33ddd5220005"
server_key_iv: "7c95a078d401c2796dfbae0479637184"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.657744 12794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:46:21.658558 12819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.658738 12794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:21.658849 12794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "573fc994e0fb429fa4b1cce2fc2b03ea"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "dce01953fa602c1e97fe33ddd5220005"
server_key_iv: "7c95a078d401c2796dfbae0479637184"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.658933 12794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:21.662317 12794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:21.662621 12794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:21.662774 12794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:21.666654 12794 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:46807
I20260808 17:46:21.666702 12903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:46807 every 8 connection(s)
I20260808 17:46:21.667039 12794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:46:21.667740 12904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:21.670768 12904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea: Bootstrap starting.
I20260808 17:46:21.671350 12904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:21.671648 12904 log.cc:824] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:21.672313 12904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea: No bootstrap required, opened a new log
I20260808 17:46:21.673241 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 12794
I20260808 17:46:21.673353 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:46:21.673543 29947 external_mini_cluster.cc:1428] Setting key f6ca3379d04a0634bdd419f7ff082a2f
I20260808 17:46:21.674223 12904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573fc994e0fb429fa4b1cce2fc2b03ea" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46807 } }
I20260808 17:46:21.674443 12904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:21.674485 12904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573fc994e0fb429fa4b1cce2fc2b03ea, State: Initialized, Role: FOLLOWER
I20260808 17:46:21.674598 12904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573fc994e0fb429fa4b1cce2fc2b03ea" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46807 } }
I20260808 17:46:21.674677 12904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:46:21.674723 12904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:46:21.674772 12904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:21.675486 12904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573fc994e0fb429fa4b1cce2fc2b03ea" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46807 } }
I20260808 17:46:21.675624 12904 leader_election.cc:302] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 573fc994e0fb429fa4b1cce2fc2b03ea; no voters:
I20260808 17:46:21.675778 12904 leader_election.cc:288] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:46:21.675949 12911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:21.676040 12904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:46:21.676097 12911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [term 1 LEADER]: Becoming Leader. State: Replica: 573fc994e0fb429fa4b1cce2fc2b03ea, State: Running, Role: LEADER
I20260808 17:46:21.676185 12911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573fc994e0fb429fa4b1cce2fc2b03ea" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46807 } }
I20260808 17:46:21.676575 12911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "573fc994e0fb429fa4b1cce2fc2b03ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573fc994e0fb429fa4b1cce2fc2b03ea" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46807 } } }
I20260808 17:46:21.676719 12911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:21.676893 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.674126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.29.62.254:46807)
0808 17:46:21.674362 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:21.674415 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:21.674452 (+ 37us) server_negotiation.cc:407] Connection header received
0808 17:46:21.674569 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:21.674576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:21.674619 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:21.674765 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:21.675805 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:21.675809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:21.675809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:46:21.675814 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:21.675863 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:46:21.675886 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:21.675963 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:21.676234 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 17:46:21.676305 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260808 17:46:21.676908 12911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 573fc994e0fb429fa4b1cce2fc2b03ea. Latest consensus state: current_term: 1 leader_uuid: "573fc994e0fb429fa4b1cce2fc2b03ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573fc994e0fb429fa4b1cce2fc2b03ea" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46807 } } }
I20260808 17:46:21.677076 12911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:21.679414 12926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:46:21.679864 12926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:46:21.681317 12926 catalog_manager.cc:1347] Generated new cluster ID: aae47adf8021488a894184e368f7538c
I20260808 17:46:21.681371 12926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:46:21.700060 12926 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:46:21.700304 12926 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:46:21.704226 12926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 573fc994e0fb429fa4b1cce2fc2b03ea: Generated new TSK 0
I20260808 17:46:21.704452 12926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:46:21.705794 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46807
--builtin_ntp_servers=127.29.62.212:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 17:46:21.790856 12941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:21.791139 12941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:21.791170 12941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:21.792379 12941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:21.792419 12941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:21.792446 12941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:46:21.793804 12941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:21.794071 12941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:21.794426 12941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:21.799628 12946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.799640 12953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.799649 12950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.800114 12941 server_base.cc:1034] running on GCE node
I20260808 17:46:21.800315 12941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:21.800570 12941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:21.801745 12941 hybrid_clock.cc:648] HybridClock initialized: now 1786211181801728 us; error 36 us; skew 500 ppm
I20260808 17:46:21.803393 12941 webserver.cc:460] Webserver started at http://127.29.62.193:39123/ using document root <none> and password file <none>
I20260808 17:46:21.803689 12941 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:21.803751 12941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:21.803895 12941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:21.805299 12941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "037e8a645bb6442cba8dc45acc6fa968"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "512bbb126d0c1fa273ea980cdd70b363"
server_key_iv: "4b079662f8b31eb99474aa64409817e5"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.805734 12941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "037e8a645bb6442cba8dc45acc6fa968"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "512bbb126d0c1fa273ea980cdd70b363"
server_key_iv: "4b079662f8b31eb99474aa64409817e5"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.807370 12941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260808 17:46:21.808169 12966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.808385 12941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:46:21.808521 12941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "037e8a645bb6442cba8dc45acc6fa968"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "512bbb126d0c1fa273ea980cdd70b363"
server_key_iv: "4b079662f8b31eb99474aa64409817e5"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.808619 12941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:21.812279 12941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:21.812567 12941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:21.812718 12941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:21.812944 12941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:21.813270 12941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:21.813326 12941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.813359 12941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:21.813386 12941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.819271 12941 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:35199
I20260808 17:46:21.819341 13150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:35199 every 8 connection(s)
I20260808 17:46:21.819692 12941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:46:21.821578 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 12941
I20260808 17:46:21.821686 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:46:21.821894 29947 external_mini_cluster.cc:1428] Setting key 7b0191384726358859c0b226f75a9949
I20260808 17:46:21.822976 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.820492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:36249 (local address 127.29.62.254:46807)
0808 17:46:21.820552 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:21.820554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:21.820781 (+ 227us) server_negotiation.cc:407] Connection header received
0808 17:46:21.821242 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:21.821243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:21.821250 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:21.821316 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:21.822709 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:21.822711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:21.822711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:46:21.822716 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:21.822737 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:46:21.822743 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:21.822800 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:21.822910 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 17:46:21.822923 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:46:21.823079 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46807
--builtin_ntp_servers=127.29.62.212:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 17:46:21.823236 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.820474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:46807 (local address 127.29.62.193:36249)
0808 17:46:21.820659 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:21.820692 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:21.821153 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:21.821367 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:21.821370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:21.821378 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:21.822565 (+ 1187us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:46:21.822566 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:21.822610 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:46:21.822642 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:21.822820 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:21.822827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:21.822831 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:46:21.822923 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 17:46:21.822998 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":106,"threads_started":1}
I20260808 17:46:21.823836 13151 heartbeater.cc:347] Connected to a master server at 127.29.62.254:46807
I20260808 17:46:21.823940 13151 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:21.824106 13151 heartbeater.cc:511] Master 127.29.62.254:46807 requested a full tablet report, sending...
I20260808 17:46:21.824718 12847 ts_manager.cc:194] Registered new tserver with Master: 037e8a645bb6442cba8dc45acc6fa968 (127.29.62.193:35199)
W20260808 17:46:21.908429 13156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:21.908659 13156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:21.908713 13156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:21.909870 13156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:21.909910 13156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:21.909936 13156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:46:21.911417 13156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:21.911638 13156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:21.911892 13156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:46:21.917054 13156 server_base.cc:1034] running on GCE node
W20260808 17:46:21.917083 13162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.917099 13166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:21.917182 13164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:21.917431 13156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:21.917719 13156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:21.918870 13156 hybrid_clock.cc:648] HybridClock initialized: now 1786211181918859 us; error 30 us; skew 500 ppm
I20260808 17:46:21.920068 13156 webserver.cc:460] Webserver started at http://127.29.62.194:45149/ using document root <none> and password file <none>
I20260808 17:46:21.920329 13156 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:21.920375 13156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:21.920570 13156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:21.921654 13156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "e7c389014f34439d87f23eab2d6d1a09"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "9eb74694e14768ed360989e3a9d9c920"
server_key_iv: "cc640bfa6f6dbca1c6a68c6400d6f624"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.922021 13156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "e7c389014f34439d87f23eab2d6d1a09"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "9eb74694e14768ed360989e3a9d9c920"
server_key_iv: "cc640bfa6f6dbca1c6a68c6400d6f624"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.923453 13156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:46:21.924237 13174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.924448 13156 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:21.924544 13156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "e7c389014f34439d87f23eab2d6d1a09"
format_stamp: "Formatted at 2026-08-08 17:46:21 on dist-test-slave-tk9z"
server_key: "9eb74694e14768ed360989e3a9d9c920"
server_key_iv: "cc640bfa6f6dbca1c6a68c6400d6f624"
server_key_version: "encryptionkey@0"
I20260808 17:46:21.924620 13156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:21.928094 13156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:21.928354 13156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:21.928496 13156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:21.928728 13156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:21.929035 13156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:21.929068 13156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.929088 13156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:21.929101 13156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:21.934818 13156 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:34589
I20260808 17:46:21.934887 13356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:34589 every 8 connection(s)
I20260808 17:46:21.935211 13156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:46:21.938323 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.935975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:40279 (local address 127.29.62.254:46807)
0808 17:46:21.936069 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:21.936071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:21.936261 (+ 190us) server_negotiation.cc:407] Connection header received
0808 17:46:21.936789 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:21.936791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:21.936799 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:21.936851 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:21.938059 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:21.938061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:21.938061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:46:21.938067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:21.938086 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:46:21.938090 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:21.938147 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:21.938233 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 17:46:21.938251 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:46:21.938546 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:21.935946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:46807 (local address 127.29.62.194:40279)
0808 17:46:21.936145 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:21.936178 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:21.936670 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:21.936926 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:21.936929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:21.936937 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:21.937901 (+ 964us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:46:21.937901 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:21.937930 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:46:21.937958 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:21.938142 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:21.938146 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:21.938150 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:46:21.938244 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 17:46:21.938306 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260808 17:46:21.938998 13359 heartbeater.cc:347] Connected to a master server at 127.29.62.254:46807
I20260808 17:46:21.939082 13359 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:21.939250 13359 heartbeater.cc:511] Master 127.29.62.254:46807 requested a full tablet report, sending...
I20260808 17:46:21.939312 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 13156
I20260808 17:46:21.939427 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:46:21.939672 29947 external_mini_cluster.cc:1428] Setting key b49d6cbecb6d42c71c23a3c983f3e30a
I20260808 17:46:21.939666 12847 ts_manager.cc:194] Registered new tserver with Master: e7c389014f34439d87f23eab2d6d1a09 (127.29.62.194:34589)
I20260808 17:46:21.940582 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46807
--builtin_ntp_servers=127.29.62.212:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 17:46:22.024508 13362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:22.024761 13362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:22.024799 13362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:22.025992 13362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:22.026067 13362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:22.026111 13362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:46:22.027604 13362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:22.027906 13362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:22.028278 13362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:22.033645 13372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:22.033622 13370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:22.033856 13362 server_base.cc:1034] running on GCE node
W20260808 17:46:22.033915 13369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:22.034186 13362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:22.034519 13362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:22.035698 13362 hybrid_clock.cc:648] HybridClock initialized: now 1786211182035686 us; error 37 us; skew 500 ppm
I20260808 17:46:22.037032 13362 webserver.cc:460] Webserver started at http://127.29.62.195:33559/ using document root <none> and password file <none>
I20260808 17:46:22.037312 13362 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:22.037385 13362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:22.037539 13362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:22.038750 13362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "abacbc4ea5c448c99c1a89f54674970d"
format_stamp: "Formatted at 2026-08-08 17:46:22 on dist-test-slave-tk9z"
server_key: "641cfec69e7da4a4f0fb9ea3de7d65cd"
server_key_iv: "a09bf86a0f1bef915a7672795386353a"
server_key_version: "encryptionkey@0"
I20260808 17:46:22.039137 13362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "abacbc4ea5c448c99c1a89f54674970d"
format_stamp: "Formatted at 2026-08-08 17:46:22 on dist-test-slave-tk9z"
server_key: "641cfec69e7da4a4f0fb9ea3de7d65cd"
server_key_iv: "a09bf86a0f1bef915a7672795386353a"
server_key_version: "encryptionkey@0"
I20260808 17:46:22.040480 13362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:46:22.041232 13387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:22.041414 13362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:22.041522 13362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "abacbc4ea5c448c99c1a89f54674970d"
format_stamp: "Formatted at 2026-08-08 17:46:22 on dist-test-slave-tk9z"
server_key: "641cfec69e7da4a4f0fb9ea3de7d65cd"
server_key_iv: "a09bf86a0f1bef915a7672795386353a"
server_key_version: "encryptionkey@0"
I20260808 17:46:22.041613 13362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:22.045068 13362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:22.045327 13362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:22.045463 13362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:22.045670 13362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:22.045991 13362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:22.046048 13362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:22.046089 13362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:22.046118 13362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:22.052232 13362 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:34479
I20260808 17:46:22.052268 13580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:34479 every 8 connection(s)
I20260808 17:46:22.052608 13362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:46:22.055707 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:22.053421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:40421 (local address 127.29.62.254:46807)
0808 17:46:22.053492 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:22.053498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:22.053729 (+ 231us) server_negotiation.cc:407] Connection header received
0808 17:46:22.054197 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:22.054198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:22.054204 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:22.054279 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:22.055480 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:22.055481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:22.055482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:46:22.055486 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:22.055501 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:46:22.055505 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:22.055556 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:22.055638 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 17:46:22.055652 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:46:22.055881 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:22.053402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:46807 (local address 127.29.62.195:40421)
0808 17:46:22.053595 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:22.053636 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:22.054104 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:22.054353 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:22.054357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:22.054399 (+ 42us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:22.055331 (+ 932us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:46:22.055331 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:22.055360 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:46:22.055389 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:22.055568 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:22.055573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:22.055575 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:46:22.055633 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:46:22.055683 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260808 17:46:22.056321 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 13362
I20260808 17:46:22.056424 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:46:22.056540 13582 heartbeater.cc:347] Connected to a master server at 127.29.62.254:46807
I20260808 17:46:22.056643 13582 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:22.056663 29947 external_mini_cluster.cc:1428] Setting key 4e36d4ecb4578e8edad1b489f4574fe7
I20260808 17:46:22.056798 13582 heartbeater.cc:511] Master 127.29.62.254:46807 requested a full tablet report, sending...
I20260808 17:46:22.057245 12847 ts_manager.cc:194] Registered new tserver with Master: abacbc4ea5c448c99c1a89f54674970d (127.29.62.195:34479)
I20260808 17:46:22.057610 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 17:46:22.059525 12908 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:46:22.059045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40238 (local address 127.29.62.254:46807)
0808 17:46:22.059127 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:22.059129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:22.059200 (+ 71us) server_negotiation.cc:407] Connection header received
0808 17:46:22.059285 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:22.059285 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:22.059290 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:22.059344 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:22.059462 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40238: BlockingRecv error: recv got EOF from 127.0.0.1:40238 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 17:46:22.059604 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 12941
I20260808 17:46:22.080462 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 13156
I20260808 17:46:22.091609 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 13362
I20260808 17:46:22.101423 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 12794
2026-08-08T17:46:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (579 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:46:22 dist-test-slave-tk9z krb5kdc[13598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:46:22 dist-test-slave-tk9z krb5kdc[13598](info): set up 2 sockets
Aug 08 17:46:22 dist-test-slave-tk9z krb5kdc[13598](info): commencing operation
krb5kdc: starting...
W20260808 17:46:23.394940 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:46:23 dist-test-slave-tk9z krb5kdc[13598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:46:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:46:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:46:23.521270 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34231
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:34231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:23.605757 13622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:23.605983 13622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:23.606007 13622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:23.607527 13622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:46:23.607584 13622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:23.607600 13622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:23.607613 13622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:46:23.607625 13622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:46:23.609290 13622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:34231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:23.609563 13622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:23.609822 13622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:23.614679 13635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:23.614699 13630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:23.614679 13632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:23.614775 13622 server_base.cc:1034] running on GCE node
I20260808 17:46:23.615115 13622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:23.615422 13622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:23.616571 13622 hybrid_clock.cc:648] HybridClock initialized: now 1786211183616552 us; error 35 us; skew 500 ppm
Aug 08 17:46:23 dist-test-slave-tk9z krb5kdc[13598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211183, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:23.619024 13622 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:46:23.619410 13622 webserver.cc:460] Webserver started at http://127.29.62.254:40409/ using document root <none> and password file <none>
I20260808 17:46:23.619606 13622 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:23.619647 13622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:23.619735 13622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:23.620738 13622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "e8a342f8911b4a09b172c4ef7115d9aa"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "4814f0baa8ed1d4f3c8e3e259e143354"
server_key_iv: "3813915dcaec988a5878fb1ad94c89fb"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.621048 13622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "e8a342f8911b4a09b172c4ef7115d9aa"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "4814f0baa8ed1d4f3c8e3e259e143354"
server_key_iv: "3813915dcaec988a5878fb1ad94c89fb"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.622308 13622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:46:23.623086 13652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:23.623262 13622 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:23.623386 13622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "e8a342f8911b4a09b172c4ef7115d9aa"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "4814f0baa8ed1d4f3c8e3e259e143354"
server_key_iv: "3813915dcaec988a5878fb1ad94c89fb"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.623466 13622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:23.643716 13622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:23.645694 13622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:23.645860 13622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:23.650043 13622 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:34231
I20260808 17:46:23.650096 13731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:34231 every 8 connection(s)
I20260808 17:46:23.650552 13622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:46:23.651631 13733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:23.653925 13733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa: Bootstrap starting.
I20260808 17:46:23.654543 13733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:23.654844 13733 log.cc:824] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:23.655570 13733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa: No bootstrap required, opened a new log
I20260808 17:46:23.657366 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 13622
I20260808 17:46:23.657478 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:46:23.657464 13733 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a342f8911b4a09b172c4ef7115d9aa" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34231 } }
I20260808 17:46:23.657652 13733 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:23.657683 29947 external_mini_cluster.cc:1428] Setting key 623eda9082c7376516a4140fb43e197e
I20260808 17:46:23.657691 13733 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a342f8911b4a09b172c4ef7115d9aa, State: Initialized, Role: FOLLOWER
I20260808 17:46:23.657843 13733 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a342f8911b4a09b172c4ef7115d9aa" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34231 } }
I20260808 17:46:23.657958 13733 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:46:23.658004 13733 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:46:23.658126 13733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:23.658835 13733 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a342f8911b4a09b172c4ef7115d9aa" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34231 } }
I20260808 17:46:23.658967 13733 leader_election.cc:302] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8a342f8911b4a09b172c4ef7115d9aa; no voters:
I20260808 17:46:23.659163 13733 leader_election.cc:288] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:46:23.659185 13744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:23.659307 13744 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [term 1 LEADER]: Becoming Leader. State: Replica: e8a342f8911b4a09b172c4ef7115d9aa, State: Running, Role: LEADER
I20260808 17:46:23.659394 13744 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a342f8911b4a09b172c4ef7115d9aa" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34231 } }
I20260808 17:46:23.659469 13733 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:46:23.659713 13744 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8a342f8911b4a09b172c4ef7115d9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a342f8911b4a09b172c4ef7115d9aa" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34231 } } }
I20260808 17:46:23.659857 13744 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:23.660070 13744 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8a342f8911b4a09b172c4ef7115d9aa. Latest consensus state: current_term: 1 leader_uuid: "e8a342f8911b4a09b172c4ef7115d9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a342f8911b4a09b172c4ef7115d9aa" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34231 } } }
I20260808 17:46:23.660181 13744 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa [sys.catalog]: This master's current role is: LEADER
I20260808 17:46:23.660516 13752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:46:23.661214 13752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:46:23 dist-test-slave-tk9z krb5kdc[13598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:46:23.666148 13752 catalog_manager.cc:1347] Generated new cluster ID: c4fb7d16e17e44429c43c08681e9ba4b
I20260808 17:46:23.666198 13752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:46:23.669628 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:23.658210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.29.62.254:34231)
0808 17:46:23.658412 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:23.658421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:23.658455 (+ 34us) server_negotiation.cc:407] Connection header received
0808 17:46:23.658791 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:23.658797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:23.658840 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:23.659000 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:23.659878 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:23.661382 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:23.662483 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:23.663075 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:23.663188 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:23.665795 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:23.665799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:23.665800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:23.665812 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:23.667651 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:23.668267 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:23.668271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:23.668271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:23.668396 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:23.668652 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:23.668653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:23.668654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:23.668943 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:23.669152 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:23.669226 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 17:46:23.669387 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260808 17:46:23.681980 13752 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:46:23.682735 13752 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:46:23.689234 13752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa: Generated new TSK 0
I20260808 17:46:23.689447 13752 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:46:23.722777 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34231
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:23.816751 13769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:23.816947 13769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:23.816972 13769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:23.818102 13769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:23.818145 13769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:23.818195 13769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:46:23.819741 13769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:23.819983 13769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:23.820209 13769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:23.825268 13782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:23.825284 13786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:23.825311 13769 server_base.cc:1034] running on GCE node
W20260808 17:46:23.825275 13784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:23.825736 13769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:23.826045 13769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:23.827275 13769 hybrid_clock.cc:648] HybridClock initialized: now 1786211183827249 us; error 45 us; skew 500 ppm
Aug 08 17:46:23 dist-test-slave-tk9z krb5kdc[13598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211183, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:23.830035 13769 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:46:23.830556 13769 webserver.cc:460] Webserver started at http://127.29.62.193:44265/ using document root <none> and password file <none>
I20260808 17:46:23.830787 13769 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:23.830834 13769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:23.830979 13769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:23.832065 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "d9ce593367439ccbad3e08a0c8225adb"
server_key_iv: "fc4e77480f9a63e08435701d8f8ec977"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.832439 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "d9ce593367439ccbad3e08a0c8225adb"
server_key_iv: "fc4e77480f9a63e08435701d8f8ec977"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.833809 13769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:46:23.834560 13803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:23.834729 13769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:46:23.834841 13769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "d9ce593367439ccbad3e08a0c8225adb"
server_key_iv: "fc4e77480f9a63e08435701d8f8ec977"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.834918 13769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:23.846470 13769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:23.848819 13769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:23.848989 13769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:23.849226 13769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:23.849556 13769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:23.849613 13769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:23.849655 13769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:23.849684 13769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:23.855773 13769 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:34597
I20260808 17:46:23.855820 13980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:34597 every 8 connection(s)
I20260808 17:46:23.856161 13769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:46:23.859541 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 13769
I20260808 17:46:23.859655 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:46:23.859902 29947 external_mini_cluster.cc:1428] Setting key f3e473194d69b6e18714228ae20870f1
Aug 08 17:46:23 dist-test-slave-tk9z krb5kdc[13598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211183, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:46:23.867398 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:23.856956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:39793 (local address 127.29.62.254:34231)
0808 17:46:23.857057 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:23.857059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:23.857235 (+ 176us) server_negotiation.cc:407] Connection header received
0808 17:46:23.857719 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:23.857720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:23.857730 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:23.857785 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:23.858784 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:23.859347 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:23.860729 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:23.860923 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:23.861024 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:23.863752 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:23.863754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:23.863754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:23.863763 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:23.865601 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:23.866174 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:23.866176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:23.866176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:23.866280 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:23.866687 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:23.866688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:23.866689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:23.866826 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:23.866905 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:23.867214 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 17:46:23.867305 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 17:46:23.867605 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:23.856908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34231 (local address 127.29.62.193:39793)
0808 17:46:23.857102 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:23.857139 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:23.857631 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:23.857854 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:23.857857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:23.858496 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:23.858697 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:23.858702 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:23.859443 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:23.859444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:23.860641 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:23.860645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:23.861041 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:23.861043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:23.861188 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:23.861707 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:23.861719 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:23.863650 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:23.865707 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:23.865716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:23.865718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:23.866085 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:23.866445 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:23.866446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:23.866447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:23.866620 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:23.866915 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:23.866923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:23.867071 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 17:46:23.867175 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 17:46:23.867352 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260808 17:46:23.868094 13981 heartbeater.cc:347] Connected to a master server at 127.29.62.254:34231
I20260808 17:46:23.868242 13981 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:23.868498 13981 heartbeater.cc:511] Master 127.29.62.254:34231 requested a full tablet report, sending...
I20260808 17:46:23.869225 13680 ts_manager.cc:194] Registered new tserver with Master: 59f0bdbb2b2f43948c0eecbb3c8b17a0 (127.29.62.193:34597)
I20260808 17:46:23.870293 13680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:39793
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:46:23.893467 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34231
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:23.978341 13988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:23.978643 13988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:23.978672 13988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:23.979889 13988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:23.979969 13988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:23.980019 13988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:46:23.981680 13988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:23.981936 13988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:23.982151 13988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:23.987347 14001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:23.987547 13988 server_base.cc:1034] running on GCE node
W20260808 17:46:23.987360 14002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:23.987370 14004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:23.987915 13988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:23.988158 13988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:23.989320 13988 hybrid_clock.cc:648] HybridClock initialized: now 1786211183989306 us; error 32 us; skew 500 ppm
Aug 08 17:46:23 dist-test-slave-tk9z krb5kdc[13598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211183, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:23.992085 13988 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:46:23.992563 13988 webserver.cc:460] Webserver started at http://127.29.62.194:33973/ using document root <none> and password file <none>
I20260808 17:46:23.992776 13988 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:23.992830 13988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:23.992928 13988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:23.994045 13988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "dcfcc8e14bc241998c1773d906865b68"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "118b3f99660cc6d8fdb5cc76ab2c2b69"
server_key_iv: "62348a03eb5d81eb6738e60e5b0a0d50"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.994463 13988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "dcfcc8e14bc241998c1773d906865b68"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "118b3f99660cc6d8fdb5cc76ab2c2b69"
server_key_iv: "62348a03eb5d81eb6738e60e5b0a0d50"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.996107 13988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:23.996878 14015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:23.997097 13988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:46:23.997246 13988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "dcfcc8e14bc241998c1773d906865b68"
format_stamp: "Formatted at 2026-08-08 17:46:23 on dist-test-slave-tk9z"
server_key: "118b3f99660cc6d8fdb5cc76ab2c2b69"
server_key_iv: "62348a03eb5d81eb6738e60e5b0a0d50"
server_key_version: "encryptionkey@0"
I20260808 17:46:23.997344 13988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:24.022492 13988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:24.024621 13988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:24.024797 13988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:24.025064 13988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:24.025447 13988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:24.025522 13988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:24.025568 13988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:24.025596 13988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:24.032024 13988 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:46815
I20260808 17:46:24.032076 14201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:46815 every 8 connection(s)
I20260808 17:46:24.032423 13988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:46:24 dist-test-slave-tk9z krb5kdc[13598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211183, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:46:24.039300 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 13988
I20260808 17:46:24.039392 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:46:24.039628 29947 external_mini_cluster.cc:1428] Setting key 3ba115b34c26ecf2d79fe65c81060143
I20260808 17:46:24.042311 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.033136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:49705 (local address 127.29.62.254:34231)
0808 17:46:24.033205 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.033207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.033439 (+ 232us) server_negotiation.cc:407] Connection header received
0808 17:46:24.033934 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.033935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.033945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.034010 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:24.035040 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.035535 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.036519 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.036705 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.036757 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.039171 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:24.039173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:24.039174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:24.039181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:24.040877 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:24.041304 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:24.041305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:24.041306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:24.041379 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:24.041645 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:24.041646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:24.041646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:24.041768 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:24.041839 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.042131 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.042221 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:46:24.042515 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.033130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34231 (local address 127.29.62.194:49705)
0808 17:46:24.033299 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:24.033336 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:24.033841 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:24.034091 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:24.034095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:24.034749 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:24.034956 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.034962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.035617 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.035618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.036441 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.036442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.036778 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.036778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.036888 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.037361 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:24.037369 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:24.039060 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:24.040975 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:24.040983 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:24.040986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:24.041223 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:24.041459 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:24.041460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:24.041460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:24.041569 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:24.041842 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:24.041850 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:24.041991 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 17:46:24.042097 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 17:46:24.042261 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260808 17:46:24.043246 14202 heartbeater.cc:347] Connected to a master server at 127.29.62.254:34231
I20260808 17:46:24.043355 14202 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:24.043546 14202 heartbeater.cc:511] Master 127.29.62.254:34231 requested a full tablet report, sending...
I20260808 17:46:24.043995 13680 ts_manager.cc:194] Registered new tserver with Master: dcfcc8e14bc241998c1773d906865b68 (127.29.62.194:46815)
I20260808 17:46:24.044418 13680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:49705
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:46:24.066998 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34231
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:46:24.152124 14210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:46:24.152335 14210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:46:24.152359 14210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:46:24.153527 14210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:46:24.153573 14210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:46:24.153600 14210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:46:24.155092 14210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:34231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:46:24.155361 14210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:46:24.155606 14210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:46:24.161823 14224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:24.161854 14228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:46:24.162032 14221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:46:24.162110 14210 server_base.cc:1034] running on GCE node
I20260808 17:46:24.162302 14210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:46:24.162562 14210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:46:24.163709 14210 hybrid_clock.cc:648] HybridClock initialized: now 1786211184163697 us; error 29 us; skew 500 ppm
Aug 08 17:46:24 dist-test-slave-tk9z krb5kdc[13598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211184, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:46:24.166159 14210 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:46:24.166642 14210 webserver.cc:460] Webserver started at http://127.29.62.195:44863/ using document root <none> and password file <none>
I20260808 17:46:24.166896 14210 fs_manager.cc:359] Metadata directory not provided
I20260808 17:46:24.166966 14210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:46:24.167124 14210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:46:24.168258 14210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "c9f9ab9c1fec4542acf90e2681cb43eb"
format_stamp: "Formatted at 2026-08-08 17:46:24 on dist-test-slave-tk9z"
server_key: "9dd13866953ca272f89608f41fea96b9"
server_key_iv: "9960dc90de6fa21f021d1ad4ab831112"
server_key_version: "encryptionkey@0"
I20260808 17:46:24.168627 14210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "c9f9ab9c1fec4542acf90e2681cb43eb"
format_stamp: "Formatted at 2026-08-08 17:46:24 on dist-test-slave-tk9z"
server_key: "9dd13866953ca272f89608f41fea96b9"
server_key_iv: "9960dc90de6fa21f021d1ad4ab831112"
server_key_version: "encryptionkey@0"
I20260808 17:46:24.170001 14210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:46:24.170837 14241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:24.171015 14210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:46:24.171118 14210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "c9f9ab9c1fec4542acf90e2681cb43eb"
format_stamp: "Formatted at 2026-08-08 17:46:24 on dist-test-slave-tk9z"
server_key: "9dd13866953ca272f89608f41fea96b9"
server_key_iv: "9960dc90de6fa21f021d1ad4ab831112"
server_key_version: "encryptionkey@0"
I20260808 17:46:24.171200 14210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:46:24.181491 14210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:46:24.183444 14210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:46:24.183640 14210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:46:24.183884 14210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:46:24.184149 14210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:46:24.184180 14210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:24.184202 14210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:46:24.184216 14210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:46:24.190316 14210 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:40593
I20260808 17:46:24.190361 14424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:40593 every 8 connection(s)
I20260808 17:46:24.190701 14210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:46:24.192775 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 14210
I20260808 17:46:24.192903 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:46:24.193076 29947 external_mini_cluster.cc:1428] Setting key b7fb124cbf168858d2bc22de35c0bc93
Aug 08 17:46:24 dist-test-slave-tk9z krb5kdc[13598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211184, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:46:24.200258 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.191392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:54749 (local address 127.29.62.254:34231)
0808 17:46:24.191453 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.191455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.191652 (+ 197us) server_negotiation.cc:407] Connection header received
0808 17:46:24.192138 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.192140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.192150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.192202 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:24.193245 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.193906 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.194976 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.195186 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.195260 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.197431 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:24.197433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:24.197433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:24.197440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:24.198735 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:24.199208 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:24.199209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:24.199210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:24.199272 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:24.199554 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:24.199555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:24.199555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:24.199670 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:24.199742 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.200031 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.200113 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:46:24.200445 14429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.191364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34231 (local address 127.29.62.195:54749)
0808 17:46:24.191526 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:24.191561 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:24.192053 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:24.192254 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:24.192257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:24.192898 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:46:24.193155 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.193163 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.193995 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.193996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.194865 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.194868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.195281 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.195282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.195408 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.195952 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:46:24.195960 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:46:24.197330 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:46:24.198832 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:24.198839 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:24.198842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:24.199110 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:24.199353 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:46:24.199354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:46:24.199354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:46:24.199476 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:46:24.199745 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:46:24.199753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:46:24.199887 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 17:46:24.199996 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 17:46:24.200154 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260808 17:46:24.200954 14427 heartbeater.cc:347] Connected to a master server at 127.29.62.254:34231
I20260808 17:46:24.201062 14427 heartbeater.cc:464] Registering TS with master...
I20260808 17:46:24.201262 14427 heartbeater.cc:511] Master 127.29.62.254:34231 requested a full tablet report, sending...
I20260808 17:46:24.201699 13680 ts_manager.cc:194] Registered new tserver with Master: c9f9ab9c1fec4542acf90e2681cb43eb (127.29.62.195:40593)
I20260808 17:46:24.202037 13680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:54749
I20260808 17:46:24.204802 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:46:24.212240 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.206105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33164 (local address 127.29.62.254:34231)
0808 17:46:24.206175 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.206177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.206272 (+ 95us) server_negotiation.cc:407] Connection header received
0808 17:46:24.206349 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.206350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.206360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.206451 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:46:24.207023 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.207502 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.208221 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.208431 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.208521 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.209741 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:46:24.209742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:46:24.209742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:46:24.209749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:46:24.211022 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:24.211388 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:24.211390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:24.211390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:24.211453 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:46:24.211670 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:46:24.211671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:46:24.211672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:46:24.211824 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:46:24.211931 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.212059 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.212140 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:46:24.213783 13680 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:46:24.215445 13680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:46:24.222103 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.217806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:40593 (local address 127.0.0.1:44072)
0808 17:46:24.218476 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:24.218486 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:24.218538 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:24.218835 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:24.218836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:24.218838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:24.218889 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.218890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.219737 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.219738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.221076 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.221079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.221756 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.221757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.221846 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.221861 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 17:46:24.221931 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 17:46:24.221996 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":48,"thread_start_us":79,"threads_started":1}
I20260808 17:46:24.222289 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.217793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:34597 (local address 127.0.0.1:51170)
0808 17:46:24.217990 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:24.218033 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:24.218103 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:24.218851 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:24.218852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:24.218853 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:24.218889 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.218890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.220189 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.220190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.221078 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.221081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.222054 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.222055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.222114 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.222118 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:46:24.222174 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 17:46:24.222215 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":62,"threads_started":1}
I20260808 17:46:24.222285 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.218047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.29.62.195:40593)
0808 17:46:24.218231 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.218237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.218602 (+ 365us) server_negotiation.cc:407] Connection header received
0808 17:46:24.218669 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.218672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.218689 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.218764 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:24.218964 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.219630 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.221183 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.221664 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.221756 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.221815 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.222144 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.222195 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260808 17:46:24.222574 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.218045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.29.62.193:34597)
0808 17:46:24.218698 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.218702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.218713 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:46:24.218751 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.218753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.218768 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.218845 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:24.218940 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.220064 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.221198 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.221933 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.222030 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.222084 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.222417 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.222471 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":90,"threads_started":1}
I20260808 17:46:24.223110 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.217880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37738 (local address 127.29.62.194:46815)
0808 17:46:24.218273 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.218277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.218285 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:46:24.218315 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.218317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.218330 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.218465 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:24.218638 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.219901 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.221171 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.221913 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.221991 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.222052 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.222972 (+ 920us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.223022 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":78,"threads_started":1}
I20260808 17:46:24.223065 14315 tablet_service.cc:1467] Processing CreateTablet for tablet 977edd0ca7c3455d8f0074bd85a423af (DEFAULT_TABLE table=test-table [id=3aa6af260d7c48b887d26697f3a3c9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:24.223218 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.217793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:46815 (local address 127.0.0.1:37738)
0808 17:46:24.218036 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:24.218046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:24.218103 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:24.218472 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:24.218475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:24.218478 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:24.218548 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.218551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.220024 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.220025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.221084 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.221086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.222088 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.222090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.222828 (+ 738us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.222835 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:46:24.222904 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 17:46:24.223104 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":5,"thread_start_us":118,"threads_started":1}
I20260808 17:46:24.223301 13876 tablet_service.cc:1467] Processing CreateTablet for tablet 977edd0ca7c3455d8f0074bd85a423af (DEFAULT_TABLE table=test-table [id=3aa6af260d7c48b887d26697f3a3c9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:24.223462 14315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977edd0ca7c3455d8f0074bd85a423af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:24.223665 13876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977edd0ca7c3455d8f0074bd85a423af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:24.226239 14455 tablet_bootstrap.cc:492] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: Bootstrap starting.
I20260808 17:46:24.226344 14456 tablet_bootstrap.cc:492] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0: Bootstrap starting.
I20260808 17:46:24.226855 14093 tablet_service.cc:1467] Processing CreateTablet for tablet 977edd0ca7c3455d8f0074bd85a423af (DEFAULT_TABLE table=test-table [id=3aa6af260d7c48b887d26697f3a3c9ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:46:24.226958 14456 tablet_bootstrap.cc:654] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:24.226986 14455 tablet_bootstrap.cc:654] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:24.227185 14093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977edd0ca7c3455d8f0074bd85a423af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:46:24.227427 14456 log.cc:824] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:24.227584 14455 log.cc:824] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:24.228200 14456 tablet_bootstrap.cc:492] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0: No bootstrap required, opened a new log
I20260808 17:46:24.228260 14456 ts_tablet_manager.cc:1397] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:46:24.228274 14455 tablet_bootstrap.cc:492] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: No bootstrap required, opened a new log
I20260808 17:46:24.228319 14455 ts_tablet_manager.cc:1397] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:46:24.229888 14455 raft_consensus.cc:348] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.230058 14455 raft_consensus.cc:374] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:24.230084 14455 raft_consensus.cc:729] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f9ab9c1fec4542acf90e2681cb43eb, State: Initialized, Role: FOLLOWER
I20260808 17:46:24.230193 14455 consensus_queue.cc:260] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.230525 14455 ts_tablet_manager.cc:1428] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:46:24.230540 14427 heartbeater.cc:503] Master 127.29.62.254:34231 was elected leader, sending a full tablet report...
I20260808 17:46:24.230963 14462 tablet_bootstrap.cc:492] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Bootstrap starting.
I20260808 17:46:24.230358 14456 raft_consensus.cc:348] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.231290 14456 raft_consensus.cc:374] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:24.231348 14456 raft_consensus.cc:729] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59f0bdbb2b2f43948c0eecbb3c8b17a0, State: Initialized, Role: FOLLOWER
I20260808 17:46:24.231463 14456 consensus_queue.cc:260] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.231629 14462 tablet_bootstrap.cc:654] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Neither blocks nor log segments found. Creating new log.
I20260808 17:46:24.231750 14456 ts_tablet_manager.cc:1428] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 17:46:24.231895 13981 heartbeater.cc:503] Master 127.29.62.254:34231 was elected leader, sending a full tablet report...
I20260808 17:46:24.232185 14462 log.cc:824] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Log is configured to *not* fsync() on all Append() calls
I20260808 17:46:24.232898 14462 tablet_bootstrap.cc:492] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: No bootstrap required, opened a new log
I20260808 17:46:24.232981 14462 ts_tablet_manager.cc:1397] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:46:24.234498 14462 raft_consensus.cc:348] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.234648 14462 raft_consensus.cc:374] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:46:24.234694 14462 raft_consensus.cc:729] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfcc8e14bc241998c1773d906865b68, State: Initialized, Role: FOLLOWER
I20260808 17:46:24.234828 14462 consensus_queue.cc:260] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.235085 14462 ts_tablet_manager.cc:1428] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:46:24.235152 14202 heartbeater.cc:503] Master 127.29.62.254:34231 was elected leader, sending a full tablet report...
W20260808 17:46:24.282982 14203 tablet.cc:2369] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:46:24.321369 14461 raft_consensus.cc:482] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:46:24.321564 14461 raft_consensus.cc:504] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.322001 14461 leader_election.cc:288] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59f0bdbb2b2f43948c0eecbb3c8b17a0 (127.29.62.193:34597), dcfcc8e14bc241998c1773d906865b68 (127.29.62.194:46815)
I20260808 17:46:24.324910 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.322346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:44045 (local address 127.29.62.193:34597)
0808 17:46:24.322418 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.322420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.322430 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:46:24.322462 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.322463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.322491 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.322570 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:24.322738 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.323214 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.324170 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.324558 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.324647 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.324657 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.324810 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.324846 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:46:24.324913 14429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.322205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:34597 (local address 127.29.62.195:44045)
0808 17:46:24.322276 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:24.322289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:24.322341 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:24.322577 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:24.322579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:24.322581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:24.322655 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.322657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.323301 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.323302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.324070 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.324073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.324646 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.324648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.324723 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.324726 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:46:24.324800 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:46:24.324840 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":27}
I20260808 17:46:24.324975 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.322346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:44691 (local address 127.29.62.194:46815)
0808 17:46:24.322419 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.322421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.322548 (+ 127us) server_negotiation.cc:407] Connection header received
0808 17:46:24.322603 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.322604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.322614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.322679 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:46:24.322846 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.323314 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.324250 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.324623 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.324709 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.324717 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.324864 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.324899 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:46:24.325029 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.322208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:46815 (local address 127.29.62.195:44691)
0808 17:46:24.322481 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:24.322489 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:46:24.322550 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:46:24.322704 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:46:24.322705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:46:24.322706 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:46:24.322779 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.322780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.323403 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.323404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.324158 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:46:24.324160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.324710 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.324711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:46:24.324788 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.324792 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:46:24.324858 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:46:24.324896 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":59,"thread_start_us":141,"threads_started":1}
I20260808 17:46:24.325352 13902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977edd0ca7c3455d8f0074bd85a423af" candidate_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" is_pre_election: true
I20260808 17:46:24.325532 13902 raft_consensus.cc:2454] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9f9ab9c1fec4542acf90e2681cb43eb in term 0.
I20260808 17:46:24.325547 14127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977edd0ca7c3455d8f0074bd85a423af" candidate_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfcc8e14bc241998c1773d906865b68" is_pre_election: true
I20260808 17:46:24.325740 14127 raft_consensus.cc:2454] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9f9ab9c1fec4542acf90e2681cb43eb in term 0.
I20260808 17:46:24.325794 14245 leader_election.cc:302] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59f0bdbb2b2f43948c0eecbb3c8b17a0, c9f9ab9c1fec4542acf90e2681cb43eb; no voters:
I20260808 17:46:24.325937 14461 raft_consensus.cc:2781] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:46:24.326026 14461 raft_consensus.cc:482] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:46:24.326054 14461 raft_consensus.cc:3037] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:24.326938 14461 raft_consensus.cc:504] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.327078 14461 leader_election.cc:288] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [CANDIDATE]: Term 1 election: Requested vote from peers 59f0bdbb2b2f43948c0eecbb3c8b17a0 (127.29.62.193:34597), dcfcc8e14bc241998c1773d906865b68 (127.29.62.194:46815)
I20260808 17:46:24.327257 14127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977edd0ca7c3455d8f0074bd85a423af" candidate_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfcc8e14bc241998c1773d906865b68"
I20260808 17:46:24.327245 13902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977edd0ca7c3455d8f0074bd85a423af" candidate_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0"
I20260808 17:46:24.327332 14127 raft_consensus.cc:3037] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:24.327356 13902 raft_consensus.cc:3037] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:46:24.327994 14127 raft_consensus.cc:2454] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9f9ab9c1fec4542acf90e2681cb43eb in term 1.
I20260808 17:46:24.328193 13902 raft_consensus.cc:2454] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9f9ab9c1fec4542acf90e2681cb43eb in term 1.
I20260808 17:46:24.328184 14242 leader_election.cc:302] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9f9ab9c1fec4542acf90e2681cb43eb, dcfcc8e14bc241998c1773d906865b68; no voters:
I20260808 17:46:24.328298 14461 raft_consensus.cc:2781] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:46:24.328505 14461 raft_consensus.cc:686] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 1 LEADER]: Becoming Leader. State: Replica: c9f9ab9c1fec4542acf90e2681cb43eb, State: Running, Role: LEADER
I20260808 17:46:24.328624 14461 consensus_queue.cc:237] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } }
I20260808 17:46:24.329516 13680 catalog_manager.cc:5495] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb reported cstate change: term changed from 0 to 1, leader changed from <none> to c9f9ab9c1fec4542acf90e2681cb43eb (127.29.62.195). New cstate: current_term: 1 leader_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9f9ab9c1fec4542acf90e2681cb43eb" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260808 17:46:24.336889 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.334799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.29.62.195:40593)
0808 17:46:24.334848 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:46:24.334850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:46:24.334857 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:46:24.334935 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:46:24.334935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:46:24.334951 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:46:24.335025 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:46:24.335126 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.335563 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.336238 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:46:24.336410 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:46:24.336459 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:46:24.336584 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:46:24.336670 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:46:24.336722 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:46:24.336755 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 17:46:24.336769 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:46:24.338683 14127 raft_consensus.cc:1261] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 1 FOLLOWER]: Refusing update from remote peer c9f9ab9c1fec4542acf90e2681cb43eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:24.338685 13902 raft_consensus.cc:1261] T 977edd0ca7c3455d8f0074bd85a423af P 59f0bdbb2b2f43948c0eecbb3c8b17a0 [term 1 FOLLOWER]: Refusing update from remote peer c9f9ab9c1fec4542acf90e2681cb43eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:46:24.338984 14461 consensus_queue.cc:1035] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcfcc8e14bc241998c1773d906865b68" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:24.339124 14468 consensus_queue.cc:1035] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "59f0bdbb2b2f43948c0eecbb3c8b17a0" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:46:24.342361 14475 mvcc.cc:205] Tried to move back new op lower bound from 7316321011048435712 to 7316321011010863104. Current Snapshot: MvccSnapshot[applied={T|T < 7316321011048435712}]
I20260808 17:46:24.343947 13679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:24.344030 13679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:46:24.344817 13679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8a342f8911b4a09b172c4ef7115d9aa: Sending DeleteTablet for 3 replicas of tablet 977edd0ca7c3455d8f0074bd85a423af
I20260808 17:46:24.345178 14315 tablet_service.cc:1514] Processing DeleteTablet for tablet 977edd0ca7c3455d8f0074bd85a423af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:24 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:44072
I20260808 17:46:24.345180 13876 tablet_service.cc:1514] Processing DeleteTablet for tablet 977edd0ca7c3455d8f0074bd85a423af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:24 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:51170
I20260808 17:46:24.345314 14093 tablet_service.cc:1514] Processing DeleteTablet for tablet 977edd0ca7c3455d8f0074bd85a423af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:46:24 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:37738
I20260808 17:46:24.345410 14495 tablet_replica.cc:330] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: stopping tablet replica
I20260808 17:46:24.345513 14495 raft_consensus.cc:2229] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:46:24.345599 14496 tablet_replica.cc:330] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: stopping tablet replica
I20260808 17:46:24.345621 14495 raft_consensus.cc:2258] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:24.345734 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 13769
I20260808 17:46:24.348287 14496 raft_consensus.cc:2229] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:46:24.348438 14496 raft_consensus.cc:2258] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:46:24.351805 14495 ts_tablet_manager.cc:1905] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:24.354090 14495 ts_tablet_manager.cc:1918] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:46:24.354166 14495 log.cc:1196] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/977edd0ca7c3455d8f0074bd85a423af
I20260808 17:46:24.354424 14495 ts_tablet_manager.cc:1939] T 977edd0ca7c3455d8f0074bd85a423af P c9f9ab9c1fec4542acf90e2681cb43eb: Deleting consensus metadata
I20260808 17:46:24.354804 13656 catalog_manager.cc:4841] TS c9f9ab9c1fec4542acf90e2681cb43eb (127.29.62.195:40593): tablet 977edd0ca7c3455d8f0074bd85a423af (table test-table [id=3aa6af260d7c48b887d26697f3a3c9ab]) successfully deleted
I20260808 17:46:24.362800 14496 ts_tablet_manager.cc:1905] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:46:24.364845 14496 ts_tablet_manager.cc:1918] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:46:24.364939 14496 log.cc:1196] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/977edd0ca7c3455d8f0074bd85a423af
I20260808 17:46:24.365235 14496 ts_tablet_manager.cc:1939] T 977edd0ca7c3455d8f0074bd85a423af P dcfcc8e14bc241998c1773d906865b68: Deleting consensus metadata
I20260808 17:46:24.365760 13654 catalog_manager.cc:4841] TS dcfcc8e14bc241998c1773d906865b68 (127.29.62.194:46815): tablet 977edd0ca7c3455d8f0074bd85a423af (table test-table [id=3aa6af260d7c48b887d26697f3a3c9ab]) successfully deleted
W20260808 17:46:24.367143 13655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:34597 (error 108)
I20260808 17:46:24.367491 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 13988
I20260808 17:46:24.369288 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:46:24.367602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:34597 (local address 127.0.0.1:51174)
0808 17:46:24.369199 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0808 17:46:24.369243 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:34597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1574}
W20260808 17:46:24.369674 13655 catalog_manager.cc:4584] TS 59f0bdbb2b2f43948c0eecbb3c8b17a0 (127.29.62.193:34597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 977edd0ca7c3455d8f0074bd85a423af: Network error: Client connection negotiation failed: client connection to 127.29.62.193:34597: connect: Connection refused (error 111)
I20260808 17:46:24.375468 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 14210
I20260808 17:46:24.387348 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 13622
2026-08-08T17:46:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2287 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:46:24 dist-test-slave-tk9z krb5kdc[14504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:46:24 dist-test-slave-tk9z krb5kdc[14504](info): set up 2 sockets
Aug 08 17:46:24 dist-test-slave-tk9z krb5kdc[14504](info): commencing operation
krb5kdc: starting...
W20260808 17:46:25.642807 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:46:25 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:46:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.62.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.62.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/rangeradmin_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/rangeradmin_127.29.62.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.62.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.62.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/rangerlookup_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/rangerlookup_127.29.62.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.62.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.62.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/HTTP_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/HTTP_127.29.62.212@KRBTEST.COM.keytab.
I20260808 17:46:25.813225 29947 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGk3gfK/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/postgres -l logfile start
2026-08-08 17:46:28.183 UTC [14549] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 17:46:28.183 UTC [14549] LOG: listening on IPv4 address "127.29.62.212", port 45631
2026-08-08 17:46:28.186 UTC [14549] LOG: listening on Unix socket "/tmp/.s.PGSQL.45631"
2026-08-08 17:46:28.208 UTC [14557] LOG: database system was shut down at 2026-08-08 17:46:26 UTC
2026-08-08 17:46:28.212 UTC [14549] LOG: database system is ready to accept connections
I20260808 17:46:29.374831 29947 mini_postgres.cc:96] Postgres bound to 45631
2026-08-08 17:46:29.383 UTC [14577] FATAL: database "slave" does not exist
127.29.62.212:45631 - accepting connections
I20260808 17:46:29.384361 29947 mini_ranger.cc:161] Starting Ranger...
I20260808 17:46:29.401758 29947 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 17:46:29.828890 29947 mini_ranger.cc:87] Created ranger Postgres database
I20260808 17:46:29.829015 29947 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-admin
2026-08-08 17:46:30,207 [I] DB FLAVOR :POSTGRES
2026-08-08 17:46:30,208 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 17:46:30,208 [I] Checking connection..
2026-08-08 17:46:30,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 17:46:30,495 [I] Checking connection passed.
2026-08-08 17:46:30,495 [I] --------- Verifying version history table ---------
2026-08-08 17:46:30,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:46:30,769 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 17:46:30,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:46:31,042 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 17:46:31,042 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 17:46:31,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 17:46:31.287 UTC [14689] WARNING: there is no transaction in progress
2026-08-08 17:46:31,297 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 17:46:31,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:46:31,534 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 17:46:31,535 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 17:46:31,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 17:46:31,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 17:46:32,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tk9z.c.gcp-upstream.internal','N') ;"
2026-08-08 17:46:32,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 17:46:32,520 [I] Table x_portal_user does not exist in database ranger
2026-08-08 17:46:32,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 17:46:32,759 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 17:46:32,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 17:46:33,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 17:46:33,266 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 17:46:33,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 17:46:34.248 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.266 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.283 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.298 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.313 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.332 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.389 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.395 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.409 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.420 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.430 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.444 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.453 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.460 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.466 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:34.473 UTC [14881] WARNING: there is no transaction in progress
2026-08-08 17:46:35,070 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 17:46:35,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 17:46:35,313 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 17:46:35,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 17:46:35,583 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 17:46:35,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 17:46:35,849 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 17:46:35,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 17:46:36,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 17:46:36,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tk9z.c.gcp-upstream.internal';"
2026-08-08 17:46:36,639 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 17:46:36,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 17:46:36,905 [I] DB_PATCHES have already been applied
I20260808 17:46:36.913797 29947 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGk3gfK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 17:46:36.913887 29947 mini_ranger.cc:193] Using host: 127.29.62.212
I20260808 17:46:36.918774 29947 mini_ranger.cc:227] Ranger admin URL: http://127.29.62.212:37279
Aug 08, 2026 5:46:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:46:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:46:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 5:46:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 5:46:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 5:46:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 5:46:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 17:46:38.273873 29947 mini_ranger.cc:160] Time spent starting Ranger: real 8.890s user 0.000s sys 0.012s
Aug 08, 2026 5:46:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 5:46:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 5:46:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 5:46:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 5:47:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 17:47:01.032039 29947 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.62.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.62.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/rangerkms_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/rangerkms_127.29.62.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.62.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.62.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.62.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/HTTP_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.62.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/HTTP_127.29.62.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 17:47:01 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211221, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 17:47:01.124763 29947 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 17:47:01.135231 29947 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 17:47:01.894214 29947 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 17:47:01.894289 29947 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms
I20260808 17:47:01.894302 29947 mini_ranger_kms.cc:227] Using postgres at 127.29.62.212:45631
2026-08-08 17:47:02,129 [I] DB FLAVOR :POSTGRES
2026-08-08 17:47:02,129 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 17:47:02,129 [I] Checking connection
2026-08-08 17:47:02,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 17:47:02,370 [I] connection success
2026-08-08 17:47:02,370 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 17:47:02,370 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 17:47:02,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 17:47:02,617 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 17:47:02,618 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 17:47:02,618 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 17:47:02,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:45631/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 17:47:02,892 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 17:47:03.392647 29947 mini_ranger_kms.cc:310] Created kms service
I20260808 17:47:03.612175 29947 mini_ranger_kms.cc:326] Created kudu user
I20260808 17:47:03.661664 29947 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 17:47:03 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211221, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:04.119266 29947 mini_ranger_kms.cc:384] Added ranger policy
I20260808 17:47:04.119398 29947 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 17:47:04.119427 29947 mini_ranger_kms.cc:240] Using host: 127.29.62.212
I20260808 17:47:04.124488 29947 mini_ranger_kms.cc:276] Ranger KMS PID: 15158
I20260808 17:47:04.124629 29947 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.62.212:55947
Aug 08, 2026 5:47:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:47:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:47:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 5:47:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 5:47:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 5:47:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 17:47:06 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786211226, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.62.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:06.937729 29947 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.813s user 0.002s sys 0.217s
I20260808 17:47:06.937860 29947 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 17:47:06.937951 29947 mini_ranger_kms.cc:401] 127.29.62.212:55947/kms/v1/keys
Aug 08 17:47:06 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211226, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.62.212@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
Aug 08 17:47:07 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211226, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.62.212@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
Aug 08 17:47:09 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:47:09.697403 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:46131
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:46131
--ranger_config_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:55947/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:47:09.782970 15246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:09.783213 15246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:09.783250 15246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:09.784425 15246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:47:09.784492 15246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:09.784514 15246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:09.784546 15246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:47:09.784569 15246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:47:09.787247 15246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.62.212:55947/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:46131
--ranger_config_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:46131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:09.787541 15246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:09.787767 15246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:09.793632 15255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:09.793630 15252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:09.793857 15246 server_base.cc:1034] running on GCE node
W20260808 17:47:09.793641 15251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:09.794114 15246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:09.794479 15246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:09.795639 15246 hybrid_clock.cc:648] HybridClock initialized: now 1786211229795609 us; error 49 us; skew 500 ppm
Aug 08 17:47:09 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211229, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:09.802700 15246 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:47:09.803242 15246 webserver.cc:460] Webserver started at http://127.29.62.254:46169/ using document root <none> and password file <none>
I20260808 17:47:09.803485 15246 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:09.803525 15246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:47:09.803668 15246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:47:09 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211229, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:09.863165 15246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0"
format_stamp: "Formatted at 2026-08-08 17:47:09 on dist-test-slave-tk9z"
server_key: "9a04fa71128035c834e1ed87053e47f3"
server_key_iv: "6881a31a7f06d1556efedcd0c2b49f16"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:09.863621 15246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0"
format_stamp: "Formatted at 2026-08-08 17:47:09 on dist-test-slave-tk9z"
server_key: "9a04fa71128035c834e1ed87053e47f3"
server_key_iv: "6881a31a7f06d1556efedcd0c2b49f16"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:09.865284 15246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:47:09.880620 15269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:09.880851 15246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:47:09.880961 15246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0"
format_stamp: "Formatted at 2026-08-08 17:47:09 on dist-test-slave-tk9z"
server_key: "9a04fa71128035c834e1ed87053e47f3"
server_key_iv: "6881a31a7f06d1556efedcd0c2b49f16"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:09.881074 15246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:09.896827 15246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:09.899008 15246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:09.899161 15246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:09.904170 15246 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:46131
I20260808 17:47:09.904198 15346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:46131 every 8 connection(s)
I20260808 17:47:09.904567 15246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:47:09.905398 15347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:47:09.908497 15347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0: Bootstrap starting.
I20260808 17:47:09.909253 15347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0: Neither blocks nor log segments found. Creating new log.
I20260808 17:47:09.909595 15347 log.cc:824] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:47:09.910408 15347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0: No bootstrap required, opened a new log
I20260808 17:47:09.913286 15347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46131 } }
I20260808 17:47:09.913497 15347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:47:09.913533 15347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c12f6d7a6bd4d1b970f1eadd5b0cad0, State: Initialized, Role: FOLLOWER
I20260808 17:47:09.913698 15347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46131 } }
I20260808 17:47:09.913784 15347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:47:09.913815 15347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:47:09.913862 15347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:47:09.914665 15347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46131 } }
I20260808 17:47:09.914711 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 15246
I20260808 17:47:09.914783 15347 leader_election.cc:302] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c12f6d7a6bd4d1b970f1eadd5b0cad0; no voters:
I20260808 17:47:09.914829 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:47:09.915024 15347 leader_election.cc:288] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:47:09.915055 29947 external_mini_cluster.cc:1428] Setting key b02ed05b38aa1fe21ecbc7ad2f146dd9
I20260808 17:47:09.915105 15351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:47:09.915361 15347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:47:09.915699 15351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [term 1 LEADER]: Becoming Leader. State: Replica: 9c12f6d7a6bd4d1b970f1eadd5b0cad0, State: Running, Role: LEADER
I20260808 17:47:09.915863 15351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46131 } }
I20260808 17:47:09.916296 15351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c12f6d7a6bd4d1b970f1eadd5b0cad0. Latest consensus state: current_term: 1 leader_uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46131 } } }
I20260808 17:47:09.916450 15351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:47:09.916565 15352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c12f6d7a6bd4d1b970f1eadd5b0cad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46131 } } }
I20260808 17:47:09.917651 15352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:47:09.917127 15347 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 17:47:09.917780 15372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:47:09.918263 15372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:47:09 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:09.930500 15372 catalog_manager.cc:1347] Generated new cluster ID: ee9217863b5b44faa079253467ad5a17
I20260808 17:47:09.930569 15372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:47:09.933619 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:09.915730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.29.62.254:46131)
0808 17:47:09.916135 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:09.916145 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:09.916179 (+ 34us) server_negotiation.cc:407] Connection header received
0808 17:47:09.916705 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:09.916714 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:09.916778 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:09.917464 (+ 686us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:09.918291 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:09.923909 (+ 5618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:09.925081 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:09.925732 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:09.926183 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:09.928855 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:09.928859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:09.928859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:09.928874 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:09.931504 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:09.931921 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:09.931924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:09.931924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:09.932032 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:09.932251 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:09.932252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:09.932253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:09.932580 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:09.933175 (+ 595us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:09.933250 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 17:47:09.933382 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":80,"threads_started":1}
I20260808 17:47:09.973345 15372 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:47:09.974169 15372 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:47:09.984143 15372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0: Generated new TSK 0
I20260808 17:47:09.984432 15372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:47:10.709452 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:55947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46131
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:47:10.817891 15417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:10.818111 15417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:10.818152 15417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:10.819474 15417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:10.819562 15417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:10.819617 15417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:47:10.821484 15417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.62.212:55947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:10.821751 15417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:10.822013 15417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:10.828773 15430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:10.828814 15433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:10.829007 15439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:10.829021 15417 server_base.cc:1034] running on GCE node
I20260808 17:47:10.829389 15417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:10.829654 15417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:10.830817 15417 hybrid_clock.cc:648] HybridClock initialized: now 1786211230830804 us; error 33 us; skew 500 ppm
Aug 08 17:47:10 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211230, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:10.833827 15417 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:47:10.834398 15417 webserver.cc:460] Webserver started at http://127.29.62.193:37221/ using document root <none> and password file <none>
I20260808 17:47:10.834726 15417 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:10.834800 15417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:47:10.834964 15417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:47:10 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211230, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:10.850427 15417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "c8b69eb7a7244c8e8dc22db866630ce6"
format_stamp: "Formatted at 2026-08-08 17:47:10 on dist-test-slave-tk9z"
server_key: "2085f26d940db05c194e00657b6f5fdd"
server_key_iv: "23e25353800c4afc48c00bd12e6fdbc4"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:10.850924 15417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "c8b69eb7a7244c8e8dc22db866630ce6"
format_stamp: "Formatted at 2026-08-08 17:47:10 on dist-test-slave-tk9z"
server_key: "2085f26d940db05c194e00657b6f5fdd"
server_key_iv: "23e25353800c4afc48c00bd12e6fdbc4"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:10.852934 15417 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 17:47:10.873049 15451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:10.873381 15417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260808 17:47:10.873555 15417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "c8b69eb7a7244c8e8dc22db866630ce6"
format_stamp: "Formatted at 2026-08-08 17:47:10 on dist-test-slave-tk9z"
server_key: "2085f26d940db05c194e00657b6f5fdd"
server_key_iv: "23e25353800c4afc48c00bd12e6fdbc4"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:10.873757 15417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:10.889577 15417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:10.892604 15417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:10.892845 15417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:10.893254 15417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:10.893754 15417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:10.893868 15417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:10.893970 15417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:10.894055 15417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:10.910410 15417 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:37937
I20260808 17:47:10.911701 15417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:47:10.917954 15629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:37937 every 8 connection(s)
I20260808 17:47:10.920327 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 15417
I20260808 17:47:10.920446 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:47:10.920710 29947 external_mini_cluster.cc:1428] Setting key 0aafd847be279a7633642a4f514575f7
Aug 08 17:47:10 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211230, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
I20260808 17:47:10.935936 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:10.921985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:46131 (local address 127.29.62.193:52973)
0808 17:47:10.922495 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:10.922546 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:10.923301 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:10.924538 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:10.924543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:10.924893 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:10.925312 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:10.925319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:10.926393 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:10.926395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:10.927752 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:10.927756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:10.928370 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:10.928371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:10.928589 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:10.929359 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:10.929370 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:10.931421 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:10.933915 (+ 2494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:10.933932 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:10.933934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:10.934323 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:10.934682 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:10.934683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:10.934683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:10.934855 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:10.935246 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:10.935253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:10.935419 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 17:47:10.935529 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 17:47:10.935695 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":91,"threads_started":1}
I20260808 17:47:10.936450 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:10.924019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:52973 (local address 127.29.62.254:46131)
0808 17:47:10.924249 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:10.924252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:10.924261 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:47:10.924295 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:10.924296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:10.924309 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:10.924398 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:10.925438 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:10.926236 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:10.927857 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:10.928208 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:10.928295 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:10.931534 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:10.931535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:10.931535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:10.931543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:10.933791 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:10.934461 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:10.934462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:10.934462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:10.934601 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:10.934986 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:10.934987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:10.934987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:10.935148 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:10.935215 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:10.936264 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0808 17:47:10.936353 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":113,"threads_started":1}
I20260808 17:47:10.936883 15630 heartbeater.cc:347] Connected to a master server at 127.29.62.254:46131
I20260808 17:47:10.937034 15630 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:10.937315 15630 heartbeater.cc:511] Master 127.29.62.254:46131 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
I20260808 17:47:10.937908 15295 ts_manager.cc:194] Registered new tserver with Master: c8b69eb7a7244c8e8dc22db866630ce6 (127.29.62.193:37937)
I20260808 17:47:10.938843 15295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:52973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:47:10.970826 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:55947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46131
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:47:11 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786211231, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 17:47:11.125912 15638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:11.126291 15638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:11.126348 15638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:11.128480 15638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:11.128551 15638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:11.128597 15638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:47:11.131636 15638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.62.212:55947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:11.132205 15638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:11.132702 15638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:11.141940 15653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:11.141950 15657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:11.142999 15652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:11.143599 15638 server_base.cc:1034] running on GCE node
I20260808 17:47:11.143941 15638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:11.144413 15638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:11.145644 15638 hybrid_clock.cc:648] HybridClock initialized: now 1786211231145599 us; error 62 us; skew 500 ppm
Aug 08 17:47:11 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211231, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:11.149703 15638 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:47:11.150346 15638 webserver.cc:460] Webserver started at http://127.29.62.194:42857/ using document root <none> and password file <none>
I20260808 17:47:11.150698 15638 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:11.150810 15638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:47:11.150990 15638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:47:11 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211231, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:11.175798 15638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "29c7bd908d484a659e2e53a6df4709b0"
format_stamp: "Formatted at 2026-08-08 17:47:11 on dist-test-slave-tk9z"
server_key: "4fbefc965d3b42af934225d4405e02e1"
server_key_iv: "79028be0ee2cfe6b480b103c1f2c16cc"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:11.176350 15638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "29c7bd908d484a659e2e53a6df4709b0"
format_stamp: "Formatted at 2026-08-08 17:47:11 on dist-test-slave-tk9z"
server_key: "4fbefc965d3b42af934225d4405e02e1"
server_key_iv: "79028be0ee2cfe6b480b103c1f2c16cc"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:11.178606 15638 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 17:47:11.193524 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:11.193801 15638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:47:11.193977 15638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "29c7bd908d484a659e2e53a6df4709b0"
format_stamp: "Formatted at 2026-08-08 17:47:11 on dist-test-slave-tk9z"
server_key: "4fbefc965d3b42af934225d4405e02e1"
server_key_iv: "79028be0ee2cfe6b480b103c1f2c16cc"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:11.194180 15638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:11.209370 15638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:11.212172 15638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:11.212347 15638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:11.212587 15638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:11.213004 15638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:11.213057 15638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:11.213092 15638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:11.213119 15638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:11.227147 15638 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:39139
I20260808 17:47:11.227705 15638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:47:11.228335 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 15638
I20260808 17:47:11.228482 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:47:11.228688 29947 external_mini_cluster.cc:1428] Setting key 6594d6bc77116885b9680ffe6a7428cb
I20260808 17:47:11.228968 15874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:39139 every 8 connection(s)
Aug 08 17:47:11 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211231, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:11.241403 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.229703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:35377 (local address 127.29.62.254:46131)
0808 17:47:11.229796 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.229800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.230177 (+ 377us) server_negotiation.cc:407] Connection header received
0808 17:47:11.231189 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.231190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.231208 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.231266 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:11.232210 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.232781 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.234672 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.234904 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.234989 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.237764 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:11.237766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:11.237767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:11.237775 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:11.239742 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:11.240250 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:11.240251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:11.240251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:11.240314 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:11.240617 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:11.240618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:11.240618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:11.240759 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:11.240817 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.241203 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.241295 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:47:11.242259 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.229692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:46131 (local address 127.29.62.194:35377)
0808 17:47:11.230054 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.230098 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:11.231066 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:11.231356 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:11.231360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:11.231752 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:11.232117 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.232124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.232905 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.232906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.234573 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.234577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.234997 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.234999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.235201 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.235894 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:11.235904 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:11.237668 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:11.239852 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:11.239860 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:11.239862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:11.240157 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:11.240394 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:11.240394 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:11.240395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:11.240534 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:11.240868 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:11.240875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:11.241037 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 17:47:11.241760 (+ 723us) client_negotiation.cc:240] Negotiation successful
0808 17:47:11.241947 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":140,"threads_started":1}
I20260808 17:47:11.244984 15877 heartbeater.cc:347] Connected to a master server at 127.29.62.254:46131
I20260808 17:47:11.245085 15877 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:11.245280 15877 heartbeater.cc:511] Master 127.29.62.254:46131 requested a full tablet report, sending...
I20260808 17:47:11.245736 15295 ts_manager.cc:194] Registered new tserver with Master: 29c7bd908d484a659e2e53a6df4709b0 (127.29.62.194:39139)
I20260808 17:47:11.246240 15295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:35377
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:47:11.273763 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:55947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46131
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:47:11.385093 15882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:11.385396 15882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:11.385438 15882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:11.386811 15882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:11.386888 15882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:11.386938 15882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:47:11.388758 15882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.62.212:55947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:46131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:11.389048 15882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:11.389315 15882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:11.395272 15894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:11.395483 15897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:11.395613 15893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:11.396651 15882 server_base.cc:1034] running on GCE node
I20260808 17:47:11.396916 15882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:11.397215 15882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:11.398451 15882 hybrid_clock.cc:648] HybridClock initialized: now 1786211231398422 us; error 39 us; skew 500 ppm
Aug 08 17:47:11 dist-test-slave-tk9z krb5kdc[14504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211231, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:11.402726 15882 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:47:11.403360 15882 webserver.cc:460] Webserver started at http://127.29.62.195:40849/ using document root <none> and password file <none>
I20260808 17:47:11.403692 15882 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:11.403771 15882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:47:11.403980 15882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:47:11 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211231, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:11.424136 15882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "c4cc218bdd4e458ca641018991e10f99"
format_stamp: "Formatted at 2026-08-08 17:47:11 on dist-test-slave-tk9z"
server_key: "c7ad4a323029596864645b12d22b7467"
server_key_iv: "0d97d734362a776c181675c1ce62c3dc"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:11.424654 15882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "c4cc218bdd4e458ca641018991e10f99"
format_stamp: "Formatted at 2026-08-08 17:47:11 on dist-test-slave-tk9z"
server_key: "c7ad4a323029596864645b12d22b7467"
server_key_iv: "0d97d734362a776c181675c1ce62c3dc"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:11.427690 15882 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 17:47:11.439631 15906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:11.439872 15882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:47:11.439962 15882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "c4cc218bdd4e458ca641018991e10f99"
format_stamp: "Formatted at 2026-08-08 17:47:11 on dist-test-slave-tk9z"
server_key: "c7ad4a323029596864645b12d22b7467"
server_key_iv: "0d97d734362a776c181675c1ce62c3dc"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:11.440043 15882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:11.460764 15882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:11.463157 15882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:11.463331 15882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:11.463599 15882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:11.463955 15882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:11.464013 15882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:11.464059 15882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:11.464083 15882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:11.475647 15882 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:45671
I20260808 17:47:11.475764 16094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:45671 every 8 connection(s)
I20260808 17:47:11.476075 15882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:47:11.483165 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 15882
I20260808 17:47:11.483306 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
Aug 08 17:47:11 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211231, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:11.483543 29947 external_mini_cluster.cc:1428] Setting key ed8760181a0373424e4e7138f8015e4d
I20260808 17:47:11.488020 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.476909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:46131 (local address 127.29.62.195:44613)
0808 17:47:11.477217 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.477285 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:11.478003 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:11.478416 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:11.478421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:11.478713 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:11.478912 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.478917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.479785 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.479786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.480695 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.480710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.481154 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.481155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.481277 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.481816 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:11.481825 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:11.483776 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:11.486503 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:11.486509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:11.486511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:11.486728 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:11.487032 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:11.487032 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:11.487032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:11.487132 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:11.487550 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:11.487554 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:11.487662 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 17:47:11.487745 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 17:47:11.487856 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":190,"threads_started":1}
I20260808 17:47:11.488188 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.476951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:44613 (local address 127.29.62.254:46131)
0808 17:47:11.477059 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.477062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.477503 (+ 441us) server_negotiation.cc:407] Connection header received
0808 17:47:11.478125 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.478126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.478137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.478280 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:11.479019 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.479691 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.480808 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.481042 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.481143 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.483868 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:11.483869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:11.483870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:11.483877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:11.486324 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:11.486814 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:11.486816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:11.486816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:11.486912 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:11.487205 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:11.487206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:11.487207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:11.487452 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:11.487581 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.487851 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.488007 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 17:47:11.488644 16096 heartbeater.cc:347] Connected to a master server at 127.29.62.254:46131
I20260808 17:47:11.488811 16096 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:11.489143 16096 heartbeater.cc:511] Master 127.29.62.254:46131 requested a full tablet report, sending...
I20260808 17:47:11.489702 15295 ts_manager.cc:194] Registered new tserver with Master: c4cc218bdd4e458ca641018991e10f99 (127.29.62.195:45671)
I20260808 17:47:11.490527 15295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:44613
I20260808 17:47:11.495883 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:47:11.506096 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.497406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.29.62.254:46131)
0808 17:47:11.497455 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.497456 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.497600 (+ 144us) server_negotiation.cc:407] Connection header received
0808 17:47:11.497726 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.497727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.497734 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.497782 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:11.498985 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.499535 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.501169 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.501350 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.501405 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.503077 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:11.503078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:11.503079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:11.503087 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:11.504782 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:11.505191 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:11.505192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:11.505193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:11.505247 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:11.505428 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:11.505429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:11.505429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:11.505550 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:11.505623 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.505905 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.505995 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:47:11.514097 15295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:47:11.517172 15295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:47:11.527675 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.521211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:37937 (local address 127.0.0.1:41768)
0808 17:47:11.522481 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.522493 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:11.522554 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:11.522844 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:11.522846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:11.522848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:47:11.522896 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.522898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.524552 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.524554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.526229 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.526233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.527363 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.527365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.527446 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.527462 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:47:11.527540 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:47:11.527594 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1250,"thread_start_us":151,"threads_started":1}
I20260808 17:47:11.527926 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.521481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.29.62.193:37937)
0808 17:47:11.521865 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.521869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.522621 (+ 752us) server_negotiation.cc:407] Connection header received
0808 17:47:11.522663 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.522666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.522684 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.522761 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:47:11.523016 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.524401 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.526348 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.527182 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.527259 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.527322 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.527813 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.527857 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":69,"threads_started":1}
I20260808 17:47:11.528445 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.521388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47360 (local address 127.29.62.195:45671)
0808 17:47:11.521744 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.521749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.522013 (+ 264us) server_negotiation.cc:407] Connection header received
0808 17:47:11.522083 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.522085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.522103 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.522183 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:47:11.522630 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.524184 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.526827 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.527588 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.527640 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.528100 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.528316 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.528360 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":113,"threads_started":1}
I20260808 17:47:11.528764 15523 tablet_service.cc:1467] Processing CreateTablet for tablet ecb766ed27274d218cbaab9397dffa8e (DEFAULT_TABLE table=test-table [id=bd0581231f634577b712d2b3c093ae64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:47:11.529165 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.520915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45671 (local address 127.0.0.1:47360)
0808 17:47:11.521910 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.521923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:11.521990 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:11.522508 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:11.522509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:11.522511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:47:11.522543 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.522544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.524286 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.524287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.526656 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.526659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.528158 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.528159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.528227 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.528231 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:47:11.529048 (+ 817us) client_negotiation.cc:240] Negotiation successful
0808 17:47:11.529092 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":714,"thread_start_us":141,"threads_started":1}
I20260808 17:47:11.529161 15523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb766ed27274d218cbaab9397dffa8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:47:11.529969 15965 tablet_service.cc:1467] Processing CreateTablet for tablet ecb766ed27274d218cbaab9397dffa8e (DEFAULT_TABLE table=test-table [id=bd0581231f634577b712d2b3c093ae64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:47:11.530357 15965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb766ed27274d218cbaab9397dffa8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:47:11.531705 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.520641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:39139 (local address 127.0.0.1:41126)
0808 17:47:11.521472 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.521517 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:11.521589 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:11.522324 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:11.522328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:11.522330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:47:11.522430 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.522433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.529139 (+ 6706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.529141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.530275 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.530279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.531450 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.531451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.531529 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.531533 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:47:11.531568 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 17:47:11.531606 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":88,"threads_started":1}
I20260808 17:47:11.532111 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.520749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.29.62.194:39139)
0808 17:47:11.522131 (+ 1382us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.522136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.522147 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:47:11.522190 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.522193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.522210 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.522327 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:47:11.524739 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.529030 (+ 4291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.530432 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.531343 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.531883 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.531968 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.531996 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.532035 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1341,"thread_start_us":102,"threads_started":1}
I20260808 17:47:11.532855 15758 tablet_service.cc:1467] Processing CreateTablet for tablet ecb766ed27274d218cbaab9397dffa8e (DEFAULT_TABLE table=test-table [id=bd0581231f634577b712d2b3c093ae64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:47:11.533186 15758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecb766ed27274d218cbaab9397dffa8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:47:11.534318 16123 tablet_bootstrap.cc:492] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6: Bootstrap starting.
I20260808 17:47:11.535058 16124 tablet_bootstrap.cc:492] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: Bootstrap starting.
I20260808 17:47:11.535567 16123 tablet_bootstrap.cc:654] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6: Neither blocks nor log segments found. Creating new log.
I20260808 17:47:11.536118 16123 log.cc:824] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6: Log is configured to *not* fsync() on all Append() calls
I20260808 17:47:11.536677 16124 tablet_bootstrap.cc:654] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: Neither blocks nor log segments found. Creating new log.
I20260808 17:47:11.537268 16124 log.cc:824] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: Log is configured to *not* fsync() on all Append() calls
I20260808 17:47:11.538014 16127 tablet_bootstrap.cc:492] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: Bootstrap starting.
I20260808 17:47:11.538336 16123 tablet_bootstrap.cc:492] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6: No bootstrap required, opened a new log
I20260808 17:47:11.538465 16123 ts_tablet_manager.cc:1397] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260808 17:47:11.539151 16127 tablet_bootstrap.cc:654] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: Neither blocks nor log segments found. Creating new log.
I20260808 17:47:11.539896 16127 log.cc:824] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:47:11.540198 16124 tablet_bootstrap.cc:492] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: No bootstrap required, opened a new log
I20260808 17:47:11.540277 16124 ts_tablet_manager.cc:1397] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 17:47:11.540761 16127 tablet_bootstrap.cc:492] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: No bootstrap required, opened a new log
I20260808 17:47:11.540848 16127 ts_tablet_manager.cc:1397] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:47:11.543229 16124 raft_consensus.cc:348] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.543440 16124 raft_consensus.cc:374] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:47:11.543486 16124 raft_consensus.cc:729] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4cc218bdd4e458ca641018991e10f99, State: Initialized, Role: FOLLOWER
I20260808 17:47:11.543756 16124 consensus_queue.cc:260] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.544129 16124 ts_tablet_manager.cc:1428] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 17:47:11.544283 16096 heartbeater.cc:503] Master 127.29.62.254:46131 was elected leader, sending a full tablet report...
I20260808 17:47:11.543756 16127 raft_consensus.cc:348] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.544420 16127 raft_consensus.cc:374] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:47:11.544461 16127 raft_consensus.cc:729] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7bd908d484a659e2e53a6df4709b0, State: Initialized, Role: FOLLOWER
I20260808 17:47:11.544590 16127 consensus_queue.cc:260] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.544848 16127 ts_tablet_manager.cc:1428] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 17:47:11.544950 15877 heartbeater.cc:503] Master 127.29.62.254:46131 was elected leader, sending a full tablet report...
I20260808 17:47:11.548411 16123 raft_consensus.cc:348] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.548645 16123 raft_consensus.cc:374] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:47:11.548738 16123 raft_consensus.cc:729] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b69eb7a7244c8e8dc22db866630ce6, State: Initialized, Role: FOLLOWER
I20260808 17:47:11.548928 16123 consensus_queue.cc:260] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.549270 16123 ts_tablet_manager.cc:1428] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6: Time spent starting tablet: real 0.011s user 0.001s sys 0.003s
I20260808 17:47:11.549430 15630 heartbeater.cc:503] Master 127.29.62.254:46131 was elected leader, sending a full tablet report...
I20260808 17:47:11.619165 16131 raft_consensus.cc:482] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:47:11.619330 16131 raft_consensus.cc:504] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.619774 16131 leader_election.cc:288] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4cc218bdd4e458ca641018991e10f99 (127.29.62.195:45671), c8b69eb7a7244c8e8dc22db866630ce6 (127.29.62.193:37937)
I20260808 17:47:11.624895 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.620133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45671 (local address 127.29.62.194:33913)
0808 17:47:11.620459 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.620471 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:11.620520 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:11.620978 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:11.620979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:11.620981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:47:11.621120 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.621123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.622278 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.622280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.623542 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.623545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.624583 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.624584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.624723 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.624727 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:47:11.624778 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:47:11.624819 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305}
I20260808 17:47:11.625444 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.620661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:33913 (local address 127.29.62.195:45671)
0808 17:47:11.620708 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.620710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.620718 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:47:11.620749 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.620750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.620759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.620810 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:47:11.621220 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.622043 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.623749 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.624351 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.624429 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.624439 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.625302 (+ 863us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.625349 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:47:11.625703 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.620896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:45449 (local address 127.29.62.193:37937)
0808 17:47:11.620943 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.620945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.621653 (+ 708us) server_negotiation.cc:407] Connection header received
0808 17:47:11.621750 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.621751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.621760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.621788 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:47:11.622071 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.622876 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.624627 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.625272 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.625321 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.625328 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.625599 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.625644 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:47:11.625743 16022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb766ed27274d218cbaab9397dffa8e" candidate_uuid: "29c7bd908d484a659e2e53a6df4709b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cc218bdd4e458ca641018991e10f99" is_pre_election: true
I20260808 17:47:11.625906 16022 raft_consensus.cc:2454] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7bd908d484a659e2e53a6df4709b0 in term 0.
I20260808 17:47:11.625901 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.620233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:37937 (local address 127.29.62.194:45449)
0808 17:47:11.621578 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.621591 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:11.621680 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:11.621846 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:11.621847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:11.621848 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:47:11.621987 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.621989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.622986 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.622987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.624536 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:11.624540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.625430 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.625431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:11.625505 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.625509 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:47:11.625779 (+ 270us) client_negotiation.cc:240] Negotiation successful
0808 17:47:11.625820 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1288,"thread_start_us":110,"threads_started":1}
I20260808 17:47:11.626138 15669 leader_election.cc:302] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7bd908d484a659e2e53a6df4709b0, c4cc218bdd4e458ca641018991e10f99; no voters:
I20260808 17:47:11.626260 16131 raft_consensus.cc:2781] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:47:11.626312 16131 raft_consensus.cc:482] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:47:11.626338 16131 raft_consensus.cc:3037] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:47:11.626748 15564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb766ed27274d218cbaab9397dffa8e" candidate_uuid: "29c7bd908d484a659e2e53a6df4709b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" is_pre_election: true
I20260808 17:47:11.626982 15564 raft_consensus.cc:2454] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c7bd908d484a659e2e53a6df4709b0 in term 0.
I20260808 17:47:11.627535 16131 raft_consensus.cc:504] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.627687 16131 leader_election.cc:288] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [CANDIDATE]: Term 1 election: Requested vote from peers c4cc218bdd4e458ca641018991e10f99 (127.29.62.195:45671), c8b69eb7a7244c8e8dc22db866630ce6 (127.29.62.193:37937)
I20260808 17:47:11.627833 16022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb766ed27274d218cbaab9397dffa8e" candidate_uuid: "29c7bd908d484a659e2e53a6df4709b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cc218bdd4e458ca641018991e10f99"
I20260808 17:47:11.627871 15564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecb766ed27274d218cbaab9397dffa8e" candidate_uuid: "29c7bd908d484a659e2e53a6df4709b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b69eb7a7244c8e8dc22db866630ce6"
I20260808 17:47:11.627914 16022 raft_consensus.cc:3037] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:47:11.627952 15564 raft_consensus.cc:3037] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:47:11.628948 16022 raft_consensus.cc:2454] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7bd908d484a659e2e53a6df4709b0 in term 1.
I20260808 17:47:11.628957 15564 raft_consensus.cc:2454] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c7bd908d484a659e2e53a6df4709b0 in term 1.
I20260808 17:47:11.629247 15669 leader_election.cc:302] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7bd908d484a659e2e53a6df4709b0, c8b69eb7a7244c8e8dc22db866630ce6; no voters:
I20260808 17:47:11.629436 16131 raft_consensus.cc:2781] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:47:11.629612 16131 raft_consensus.cc:686] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 1 LEADER]: Becoming Leader. State: Replica: 29c7bd908d484a659e2e53a6df4709b0, State: Running, Role: LEADER
I20260808 17:47:11.629704 16131 consensus_queue.cc:237] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } }
I20260808 17:47:11.630645 15295 catalog_manager.cc:5495] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c7bd908d484a659e2e53a6df4709b0 (127.29.62.194). New cstate: current_term: 1 leader_uuid: "29c7bd908d484a659e2e53a6df4709b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7bd908d484a659e2e53a6df4709b0" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39139 } health_report { overall_health: HEALTHY } } }
I20260808 17:47:11.643643 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.640176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.29.62.194:39139)
0808 17:47:11.640220 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:11.640222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:11.640230 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:47:11.640337 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:11.640338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:11.640359 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:11.640423 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:47:11.640630 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.641373 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.642655 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:11.642954 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:11.643050 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:11.643288 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:47:11.643400 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:47:11.643500 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:11.643539 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 17:47:11.643560 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:47:11.645923 16022 raft_consensus.cc:1261] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 1 FOLLOWER]: Refusing update from remote peer 29c7bd908d484a659e2e53a6df4709b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:47:11.645923 15564 raft_consensus.cc:1261] T ecb766ed27274d218cbaab9397dffa8e P c8b69eb7a7244c8e8dc22db866630ce6 [term 1 FOLLOWER]: Refusing update from remote peer 29c7bd908d484a659e2e53a6df4709b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:47:11.646340 16135 consensus_queue.cc:1035] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8b69eb7a7244c8e8dc22db866630ce6" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 37937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:47:11.646469 16131 consensus_queue.cc:1035] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4cc218bdd4e458ca641018991e10f99" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:47:11.650206 16138 mvcc.cc:205] Tried to move back new op lower bound from 7316321204818620416 to 7316321204755890176. Current Snapshot: MvccSnapshot[applied={T|T < 7316321204818620416}]
I20260808 17:47:11.650890 16140 mvcc.cc:205] Tried to move back new op lower bound from 7316321204818620416 to 7316321204755890176. Current Snapshot: MvccSnapshot[applied={T|T < 7316321204818620416}]
I20260808 17:47:11.653072 16139 mvcc.cc:205] Tried to move back new op lower bound from 7316321204818620416 to 7316321204755890176. Current Snapshot: MvccSnapshot[applied={T|T < 7316321204818620416}]
I20260808 17:47:11.656004 15294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:47:11.656176 15294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:47:11.657096 15294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c12f6d7a6bd4d1b970f1eadd5b0cad0: Sending DeleteTablet for 3 replicas of tablet ecb766ed27274d218cbaab9397dffa8e
I20260808 17:47:11.657569 15523 tablet_service.cc:1514] Processing DeleteTablet for tablet ecb766ed27274d218cbaab9397dffa8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:47:11 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:41768
I20260808 17:47:11.658015 15965 tablet_service.cc:1514] Processing DeleteTablet for tablet ecb766ed27274d218cbaab9397dffa8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:47:11 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:47360
I20260808 17:47:11.658421 15758 tablet_service.cc:1514] Processing DeleteTablet for tablet ecb766ed27274d218cbaab9397dffa8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:47:11 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:41126
I20260808 17:47:11.659129 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 15417
I20260808 17:47:11.674851 16164 tablet_replica.cc:330] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: stopping tablet replica
I20260808 17:47:11.675125 16165 tablet_replica.cc:330] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: stopping tablet replica
I20260808 17:47:11.675359 16165 raft_consensus.cc:2229] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:47:11.675549 16164 raft_consensus.cc:2229] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:47:11.675819 16165 raft_consensus.cc:2258] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:47:11.676007 16164 raft_consensus.cc:2258] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:47:11.681864 16165 ts_tablet_manager.cc:1905] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:47:11.683199 16164 ts_tablet_manager.cc:1905] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:47:11.686105 16164 ts_tablet_manager.cc:1918] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:47:11.686877 16165 ts_tablet_manager.cc:1918] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:47:11.687028 16164 log.cc:1196] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/ecb766ed27274d218cbaab9397dffa8e
I20260808 17:47:11.687758 16164 ts_tablet_manager.cc:1939] T ecb766ed27274d218cbaab9397dffa8e P c4cc218bdd4e458ca641018991e10f99: Deleting consensus metadata
I20260808 17:47:11.687906 16165 log.cc:1196] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/ecb766ed27274d218cbaab9397dffa8e
I20260808 17:47:11.688271 16165 ts_tablet_manager.cc:1939] T ecb766ed27274d218cbaab9397dffa8e P 29c7bd908d484a659e2e53a6df4709b0: Deleting consensus metadata
I20260808 17:47:11.688804 15273 catalog_manager.cc:4841] TS 29c7bd908d484a659e2e53a6df4709b0 (127.29.62.194:39139): tablet ecb766ed27274d218cbaab9397dffa8e (table test-table [id=bd0581231f634577b712d2b3c093ae64]) successfully deleted
W20260808 17:47:11.690385 15270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:37937 (error 108)
I20260808 17:47:11.692416 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 15638
I20260808 17:47:11.713012 15270 catalog_manager.cc:4841] TS c4cc218bdd4e458ca641018991e10f99 (127.29.62.195:45671): tablet ecb766ed27274d218cbaab9397dffa8e (table test-table [id=bd0581231f634577b712d2b3c093ae64]) successfully deleted
I20260808 17:47:11.718644 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.718450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:37937 (local address 127.0.0.1:41770)
0808 17:47:11.718538 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.718594 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 17:47:11.718931 15270 catalog_manager.cc:4584] TS c8b69eb7a7244c8e8dc22db866630ce6 (127.29.62.193:37937): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ecb766ed27274d218cbaab9397dffa8e: Network error: Client connection negotiation failed: client connection to 127.29.62.193:37937: connect: Connection refused (error 111)
I20260808 17:47:11.735433 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 15882
Aug 08 17:47:11 dist-test-slave-tk9z krb5kdc[14504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211231, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:11.775959 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.775808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:37937 (local address 127.0.0.1:41772)
0808 17:47:11.775865 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.775900 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:47:11.776834 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:11.776675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:37937 (local address 127.0.0.1:41778)
0808 17:47:11.776764 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:11.776792 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 17:47:11.781476 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 15246
I20260808 17:47:11.801921 29947 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 17:47:12.146713 29947 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 17:47:12.146857 29947 mini_ranger.cc:66] Stopping Ranger...
I20260808 17:47:12.501490 29947 mini_ranger.cc:68] Stopped Ranger
2026-08-08 17:47:12.501 UTC [14549] LOG: received smart shutdown request
2026-08-08 17:47:12.505 UTC [14549] LOG: background worker "logical replication launcher" (PID 14575) exited with exit code 1
2026-08-08 17:47:12.505 UTC [14561] LOG: shutting down
2026-08-08 17:47:12.638 UTC [14549] LOG: database system is shut down
2026-08-08T17:47:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48358 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:47:12 dist-test-slave-tk9z krb5kdc[16184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:47:12 dist-test-slave-tk9z krb5kdc[16184](info): set up 2 sockets
Aug 08 17:47:12 dist-test-slave-tk9z krb5kdc[16184](info): commencing operation
krb5kdc: starting...
W20260808 17:47:13.989733 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:47:14 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:47:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.62.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.62.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/rangeradmin_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/rangeradmin_127.29.62.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.62.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.62.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/rangerlookup_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/rangerlookup_127.29.62.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.62.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.62.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/HTTP_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/HTTP_127.29.62.212@KRBTEST.COM.keytab.
I20260808 17:47:14.160362 29947 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGk3gfK/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/postgres -l logfile start
2026-08-08 17:47:16.469 UTC [16233] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 17:47:16.469 UTC [16233] LOG: listening on IPv4 address "127.29.62.212", port 51633
2026-08-08 17:47:16.472 UTC [16233] LOG: listening on Unix socket "/tmp/.s.PGSQL.51633"
2026-08-08 17:47:16.493 UTC [16239] LOG: database system was shut down at 2026-08-08 17:47:15 UTC
2026-08-08 17:47:16.497 UTC [16233] LOG: database system is ready to accept connections
I20260808 17:47:17.644603 29947 mini_postgres.cc:96] Postgres bound to 51633
2026-08-08 17:47:17.652 UTC [16251] FATAL: database "slave" does not exist
127.29.62.212:51633 - accepting connections
I20260808 17:47:17.652681 29947 mini_ranger.cc:161] Starting Ranger...
I20260808 17:47:17.668496 29947 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 17:47:18.098563 29947 mini_ranger.cc:87] Created ranger Postgres database
I20260808 17:47:18.098642 29947 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-admin
2026-08-08 17:47:18,399 [I] DB FLAVOR :POSTGRES
2026-08-08 17:47:18,399 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 17:47:18,399 [I] Checking connection..
2026-08-08 17:47:18,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 17:47:18,674 [I] Checking connection passed.
2026-08-08 17:47:18,674 [I] --------- Verifying version history table ---------
2026-08-08 17:47:18,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:47:18,954 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 17:47:18,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:47:19,218 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 17:47:19,218 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 17:47:19,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 17:47:19.465 UTC [16355] WARNING: there is no transaction in progress
2026-08-08 17:47:19,473 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 17:47:19,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 17:47:19,720 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 17:47:19,721 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 17:47:19,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 17:47:19,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 17:47:20,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tk9z.c.gcp-upstream.internal','N') ;"
2026-08-08 17:47:20,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 17:47:20,724 [I] Table x_portal_user does not exist in database ranger
2026-08-08 17:47:20,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 17:47:20,992 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 17:47:20,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 17:47:21,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 17:47:21,489 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 17:47:21,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 17:47:22.445 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.461 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.476 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.490 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.504 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.522 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.574 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.580 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.593 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.603 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.613 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.626 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.635 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.641 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.646 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:22.652 UTC [16529] WARNING: there is no transaction in progress
2026-08-08 17:47:23,225 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 17:47:23,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 17:47:23,460 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 17:47:23,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 17:47:23,727 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 17:47:23,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 17:47:23,989 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 17:47:23,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 17:47:24,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 17:47:24,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tk9z.c.gcp-upstream.internal';"
2026-08-08 17:47:24,732 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 17:47:24,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 17:47:24,969 [I] DB_PATCHES have already been applied
I20260808 17:47:24.975545 29947 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGk3gfK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 17:47:24.975615 29947 mini_ranger.cc:193] Using host: 127.29.62.212
I20260808 17:47:24.979638 29947 mini_ranger.cc:227] Ranger admin URL: http://127.29.62.212:37483
Aug 08, 2026 5:47:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:47:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:47:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 5:47:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 5:47:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 5:47:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 5:47:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 17:47:26.350752 29947 mini_ranger.cc:160] Time spent starting Ranger: real 8.698s user 0.003s sys 0.008s
Aug 08, 2026 5:47:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 5:47:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 5:47:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 5:47:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 5:47:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 5:47:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 5:47:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 5:47:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 17:47:48.541594 29947 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.62.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.62.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/rangerkms_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.62.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/rangerkms_127.29.62.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.62.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.62.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.62.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/HTTP_127.29.62.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.62.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/HTTP_127.29.62.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 17:47:48 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211268, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 17:47:48.644752 29947 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 17:47:48.655951 29947 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 17:47:49.422807 29947 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 17:47:49.422888 29947 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms
I20260808 17:47:49.422902 29947 mini_ranger_kms.cc:227] Using postgres at 127.29.62.212:51633
2026-08-08 17:47:49,605 [I] DB FLAVOR :POSTGRES
2026-08-08 17:47:49,605 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 17:47:49,605 [I] Checking connection
2026-08-08 17:47:49,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 17:47:49,858 [I] connection success
2026-08-08 17:47:49,858 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 17:47:49,859 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 17:47:49,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 17:47:50,110 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 17:47:50,110 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 17:47:50,110 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 17:47:50,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.62.212:51633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 17:47:50,369 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 17:47:50.864750 29947 mini_ranger_kms.cc:310] Created kms service
I20260808 17:47:51.073787 29947 mini_ranger_kms.cc:326] Created kudu user
I20260808 17:47:51.117285 29947 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 17:47:51 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211268, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:51.549053 29947 mini_ranger_kms.cc:384] Added ranger policy
I20260808 17:47:51.549271 29947 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGk3gfK/build/release/bin/postgresql.jar:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGk3gfK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 17:47:51.549299 29947 mini_ranger_kms.cc:240] Using host: 127.29.62.212
I20260808 17:47:51.554106 29947 mini_ranger_kms.cc:276] Ranger KMS PID: 16831
I20260808 17:47:51.554201 29947 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.62.212:33487
Aug 08, 2026 5:47:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 5:47:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 5:47:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 5:47:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 5:47:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 5:47:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 17:47:53 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786211273, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.62.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:54.319239 29947 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.674s user 0.005s sys 0.162s
I20260808 17:47:54.319348 29947 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 17:47:54.319396 29947 mini_ranger_kms.cc:401] 127.29.62.212:33487/kms/v1/keys
Aug 08 17:47:54 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211273, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.62.212@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
Aug 08 17:47:54 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211273, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.62.212@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
Aug 08 17:47:57 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:47:57.193679 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:45003
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:45003
--ranger_config_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:33487/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:47:57.274586 16914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:57.274817 16914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:57.274854 16914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:57.276013 16914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:47:57.276081 16914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:57.276106 16914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:57.276129 16914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:47:57.276157 16914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:47:57.277659 16914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.62.212:33487/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:45003
--ranger_config_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:45003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:57.277933 16914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:57.278159 16914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:57.283221 16923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:57.283236 16924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:57.283248 16914 server_base.cc:1034] running on GCE node
W20260808 17:47:57.283213 16928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:57.283669 16914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:57.283968 16914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:57.285142 16914 hybrid_clock.cc:648] HybridClock initialized: now 1786211277285128 us; error 27 us; skew 500 ppm
Aug 08 17:47:57 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211277, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:57.290901 16914 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:47:57.291343 16914 webserver.cc:460] Webserver started at http://127.29.62.254:45631/ using document root <none> and password file <none>
I20260808 17:47:57.291568 16914 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:57.291628 16914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:47:57.291788 16914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:47:57 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211277, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:57.339022 16914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "95ba15cdaa434cc4ad886ae4d3be6d10"
format_stamp: "Formatted at 2026-08-08 17:47:57 on dist-test-slave-tk9z"
server_key: "c6994dd4ecf12a7a90998b2503da2f78"
server_key_iv: "8fc6ace16961b6d6be16bcfc5a783674"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:57.339545 16914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "95ba15cdaa434cc4ad886ae4d3be6d10"
format_stamp: "Formatted at 2026-08-08 17:47:57 on dist-test-slave-tk9z"
server_key: "c6994dd4ecf12a7a90998b2503da2f78"
server_key_iv: "8fc6ace16961b6d6be16bcfc5a783674"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:57.341076 16914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:47:57.352818 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:57.353039 16914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:47:57.353142 16914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "95ba15cdaa434cc4ad886ae4d3be6d10"
format_stamp: "Formatted at 2026-08-08 17:47:57 on dist-test-slave-tk9z"
server_key: "c6994dd4ecf12a7a90998b2503da2f78"
server_key_iv: "8fc6ace16961b6d6be16bcfc5a783674"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:57.353256 16914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:57.379571 16914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:57.381886 16914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:57.382069 16914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:57.387575 16914 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:45003
I20260808 17:47:57.387666 17023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:45003 every 8 connection(s)
I20260808 17:47:57.388077 16914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:47:57.388844 17024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:47:57.390120 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 16914
I20260808 17:47:57.390241 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:47:57.390497 29947 external_mini_cluster.cc:1428] Setting key ecb367fec6db0050bab3a10f29f00552
I20260808 17:47:57.391933 17024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Bootstrap starting.
I20260808 17:47:57.392640 17024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Neither blocks nor log segments found. Creating new log.
I20260808 17:47:57.392963 17024 log.cc:824] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Log is configured to *not* fsync() on all Append() calls
I20260808 17:47:57.393849 17024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: No bootstrap required, opened a new log
I20260808 17:47:57.396085 17024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } }
I20260808 17:47:57.396363 17024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:47:57.396436 17024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ba15cdaa434cc4ad886ae4d3be6d10, State: Initialized, Role: FOLLOWER
I20260808 17:47:57.396593 17024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } }
I20260808 17:47:57.396700 17024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:47:57.396757 17024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:47:57.396831 17024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:47:57.397423 17024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } }
I20260808 17:47:57.397547 17024 leader_election.cc:302] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95ba15cdaa434cc4ad886ae4d3be6d10; no voters:
I20260808 17:47:57.397732 17024 leader_election.cc:288] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:47:57.397755 17031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:47:57.397985 17031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 1 LEADER]: Becoming Leader. State: Replica: 95ba15cdaa434cc4ad886ae4d3be6d10, State: Running, Role: LEADER
I20260808 17:47:57.398121 17031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } }
I20260808 17:47:57.398159 17024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:47:57.398563 17031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } } }
Aug 08 17:47:57 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:57.398733 17031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: This master's current role is: LEADER
I20260808 17:47:57.398921 17031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95ba15cdaa434cc4ad886ae4d3be6d10. Latest consensus state: current_term: 1 leader_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } } }
I20260808 17:47:57.399048 17031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: This master's current role is: LEADER
I20260808 17:47:57.399223 17045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:47:57.399813 17024 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 17:47:57.400236 17045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:47:57.407367 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:57.391214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.29.62.254:45003)
0808 17:47:57.391393 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:57.391404 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:57.391444 (+ 40us) server_negotiation.cc:407] Connection header received
0808 17:47:57.391994 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:57.392004 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:57.392068 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:57.392224 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:57.393044 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:57.394359 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:57.395496 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:57.396116 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:57.396230 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:57.399184 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:57.399188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:57.399188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:57.399201 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:57.404641 (+ 5440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:57.405176 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:57.405180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:57.405181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:57.405343 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:57.405631 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:57.405633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:57.405633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:57.406241 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:57.406481 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:57.406970 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 17:47:57.407144 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260808 17:47:57.412966 17045 catalog_manager.cc:1347] Generated new cluster ID: 543c733d491644c0b41eca44c9e5edab
I20260808 17:47:57.413035 17045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:47:57.431821 17045 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:47:57.432600 17045 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:47:57.439177 17045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Generated new TSK 0
I20260808 17:47:57.439368 17045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:47:58.105288 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:33487/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45003
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:47:58.222209 17092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:58.222487 17092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:58.222517 17092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:58.223662 17092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:58.223707 17092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:58.223735 17092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:47:58.225207 17092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.62.212:33487/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:58.225436 17092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:58.225699 17092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:58.230701 17102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:58.230748 17098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:58.230825 17092 server_base.cc:1034] running on GCE node
W20260808 17:47:58.230924 17100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:58.231165 17092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:58.231439 17092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:58.232610 17092 hybrid_clock.cc:648] HybridClock initialized: now 1786211278232597 us; error 32 us; skew 500 ppm
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:58.235392 17092 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:47:58.235865 17092 webserver.cc:460] Webserver started at http://127.29.62.193:42051/ using document root <none> and password file <none>
I20260808 17:47:58.236129 17092 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:58.236202 17092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:47:58.236361 17092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:58.249547 17092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "da0078e49efd4405b0bc9604d23e433d"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "5ca02a38610d2f3d07a389888a3e24b0"
server_key_iv: "6ba8937d95886a74839c988f36b2170a"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.250149 17092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "da0078e49efd4405b0bc9604d23e433d"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "5ca02a38610d2f3d07a389888a3e24b0"
server_key_iv: "6ba8937d95886a74839c988f36b2170a"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.252141 17092 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 17:47:58.266036 17112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.266245 17092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:47:58.266374 17092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "da0078e49efd4405b0bc9604d23e433d"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "5ca02a38610d2f3d07a389888a3e24b0"
server_key_iv: "6ba8937d95886a74839c988f36b2170a"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.266562 17092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:58.278792 17092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:58.281620 17092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:58.281804 17092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:58.282076 17092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:58.282608 17092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:58.282683 17092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.282733 17092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:58.282768 17092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.290501 17092 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:46665
I20260808 17:47:58.290585 17302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:46665 every 8 connection(s)
I20260808 17:47:58.291056 17092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:47:58.292817 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 17092
I20260808 17:47:58.292917 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:47:58.293114 29947 external_mini_cluster.cc:1428] Setting key 768a00124b2705172d89a3a2a0140e9a
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
I20260808 17:47:58.307868 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:58.292223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45003 (local address 127.29.62.193:53805)
0808 17:47:58.297137 (+ 4914us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:58.297177 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:58.297800 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:58.298113 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:58.298117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:58.298464 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:58.298839 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:58.298847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.299946 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.299947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:58.301217 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:58.301220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.301649 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.301651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:58.301779 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:58.302406 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:58.302417 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:58.304101 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:58.306226 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:58.306233 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:58.306235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:58.306561 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:58.306784 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:58.306785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:58.306785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:58.306907 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:58.307215 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:58.307222 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:58.307378 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 17:47:58.307476 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 17:47:58.307630 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4871,"thread_start_us":87,"threads_started":1}
I20260808 17:47:58.308359 17309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:58.292528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:53805 (local address 127.29.62.254:45003)
0808 17:47:58.296009 (+ 3481us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:58.296011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:58.297280 (+ 1269us) server_negotiation.cc:407] Connection header received
0808 17:47:58.297904 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:58.297905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:58.297920 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:58.297994 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:58.298952 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.299811 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.301321 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.301534 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.301613 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:58.304214 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:58.304216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:58.304216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:58.304224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:58.306131 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:58.306648 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:58.306649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:58.306650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:58.306715 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:58.306990 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:58.306991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:58.306991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:58.307125 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:58.307187 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:58.308171 (+ 984us) server_negotiation.cc:299] Negotiation successful
0808 17:47:58.308260 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3416,"thread_start_us":83,"threads_started":1}
I20260808 17:47:58.308961 17304 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45003
I20260808 17:47:58.309077 17304 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:58.309262 17304 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:47:58.309865 16968 ts_manager.cc:194] Registered new tserver with Master: da0078e49efd4405b0bc9604d23e433d (127.29.62.193:46665)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
I20260808 17:47:58.310797 16968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:53805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:47:58.327721 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:33487/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45003
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:47:58.439858 17313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:58.440151 17313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:58.440192 17313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:58.441977 17313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:58.442045 17313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:58.442088 17313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:47:58.444378 17313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.62.212:33487/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:58.444671 17313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:58.445000 17313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:58.450352 17325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:58.450511 17327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:58.450384 17323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:58.450781 17313 server_base.cc:1034] running on GCE node
I20260808 17:47:58.451010 17313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:58.451257 17313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:58.452445 17313 hybrid_clock.cc:648] HybridClock initialized: now 1786211278452425 us; error 34 us; skew 500 ppm
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:58.455107 17313 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:47:58.455600 17313 webserver.cc:460] Webserver started at http://127.29.62.194:41805/ using document root <none> and password file <none>
I20260808 17:47:58.455866 17313 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:58.455937 17313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:47:58.456097 17313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:58.468369 17313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "e8cc232cc5484e159a21c4661ba2177e"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "3f9dc976c12cc99ad4844a2bb9ae1b8a"
server_key_iv: "eed6dc71ee3d125036c4d6521cb046c6"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.468832 17313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "e8cc232cc5484e159a21c4661ba2177e"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "3f9dc976c12cc99ad4844a2bb9ae1b8a"
server_key_iv: "eed6dc71ee3d125036c4d6521cb046c6"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.470806 17313 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:58.490784 17338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.491014 17313 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260808 17:47:58.491127 17313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "e8cc232cc5484e159a21c4661ba2177e"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "3f9dc976c12cc99ad4844a2bb9ae1b8a"
server_key_iv: "eed6dc71ee3d125036c4d6521cb046c6"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.491253 17313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:58.542838 17313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:58.545361 17313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:58.545521 17313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:58.545764 17313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:58.546141 17313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:58.546240 17313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.546314 17313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:58.546448 17313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.553619 17313 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:46803
I20260808 17:47:58.554107 17313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:47:58.554482 17511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:46803 every 8 connection(s)
I20260808 17:47:58.560207 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 17313
I20260808 17:47:58.560315 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:47:58.560554 29947 external_mini_cluster.cc:1428] Setting key 15b7e35ceb06e3b0feae6001938431a0
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:58.569545 17309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:58.555173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:55853 (local address 127.29.62.254:45003)
0808 17:47:58.555258 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:58.555261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:58.558658 (+ 3397us) server_negotiation.cc:407] Connection header received
0808 17:47:58.559357 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:58.559358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:58.559375 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:58.559454 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:58.560268 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.561042 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.562974 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.563270 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.563321 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:58.566072 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:58.566074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:58.566074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:58.566081 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:58.567854 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:58.568373 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:58.568374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:58.568375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:58.568446 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:58.568747 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:58.568748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:58.568748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:58.568882 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:58.568924 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:58.569338 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 17:47:58.569431 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 17:47:58.570325 17515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:58.554879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45003 (local address 127.29.62.194:55853)
0808 17:47:58.558495 (+ 3616us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:58.558548 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:58.559229 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:58.559567 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:58.559571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:58.559821 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:58.560138 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:58.560146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.561539 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.561540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:58.562859 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:58.562863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.563436 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.563437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:58.563595 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:58.564219 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:58.564230 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:58.565954 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:58.567956 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:58.567963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:58.567965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:58.568272 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:58.568535 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:58.568535 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:58.568536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:58.568670 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:58.568969 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:58.568975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:58.569183 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 17:47:58.569890 (+ 707us) client_negotiation.cc:240] Negotiation successful
0808 17:47:58.570059 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3560,"thread_start_us":86,"threads_started":1}
I20260808 17:47:58.571028 17512 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45003
I20260808 17:47:58.571127 17512 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:58.571322 17512 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:47:58.571870 16968 ts_manager.cc:194] Registered new tserver with Master: e8cc232cc5484e159a21c4661ba2177e (127.29.62.194:46803)
I20260808 17:47:58.572335 16968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:55853
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:47:58.593781 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:33487/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45003
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:47:58.713390 17522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:58.713614 17522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:58.713651 17522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:58.714857 17522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:58.714922 17522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:58.714967 17522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:47:58.716454 17522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.62.212:33487/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:58.716727 17522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:58.716975 17522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:58.722003 17529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:58.722010 17530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:58.722240 17522 server_base.cc:1034] running on GCE node
W20260808 17:47:58.722024 17534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:58.722656 17522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:58.722918 17522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:58.724087 17522 hybrid_clock.cc:648] HybridClock initialized: now 1786211278724065 us; error 41 us; skew 500 ppm
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:58.726737 17522 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:47:58.727231 17522 webserver.cc:460] Webserver started at http://127.29.62.195:36695/ using document root <none> and password file <none>
I20260808 17:47:58.727481 17522 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:58.727550 17522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:47:58.727699 17522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:58.747726 17522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "d2f42bb62118477faa85b87adfccd528"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "884fa11d4357279db009cbeb25ee9ed9"
server_key_iv: "c960367faf786d03757050c7484bf565"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.748196 17522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "d2f42bb62118477faa85b87adfccd528"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "884fa11d4357279db009cbeb25ee9ed9"
server_key_iv: "c960367faf786d03757050c7484bf565"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.750296 17522 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 17:47:58.776042 17543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.776446 17522 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.001s
I20260808 17:47:58.776556 17522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "d2f42bb62118477faa85b87adfccd528"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "884fa11d4357279db009cbeb25ee9ed9"
server_key_iv: "c960367faf786d03757050c7484bf565"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:58.776660 17522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:58.796022 17522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:58.798489 17522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:58.798646 17522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:58.798848 17522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:58.799237 17522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:58.799304 17522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.799340 17522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:58.799367 17522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:58.806614 17522 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:36067
I20260808 17:47:58.807080 17522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:47:58.807108 17719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:36067 every 8 connection(s)
Aug 08 17:47:58 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211278, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:58.816879 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 17522
I20260808 17:47:58.816998 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:47:58.817198 29947 external_mini_cluster.cc:1428] Setting key a2658b37697d0db79a23e1c10fc4b4f3
I20260808 17:47:58.818990 17309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:58.808026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:59741 (local address 127.29.62.254:45003)
0808 17:47:58.808072 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:58.808074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:58.808368 (+ 294us) server_negotiation.cc:407] Connection header received
0808 17:47:58.809053 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:58.809055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:58.809068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:58.809100 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:58.809945 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.810765 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.812248 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.812577 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.812662 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:58.815214 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:58.815216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:58.815216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:58.815224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:58.816978 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:58.817616 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:58.817617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:58.817617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:58.817679 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:58.818196 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:58.818197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:58.818197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:58.818336 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:58.818418 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:58.818811 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 17:47:58.818898 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:47:58.819521 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:58.807999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45003 (local address 127.29.62.195:59741)
0808 17:47:58.808242 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:58.808282 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:58.808917 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:58.809292 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:58.809296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:58.809555 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:58.809823 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:58.809830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.810886 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.810887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:58.812148 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:58.812152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:58.812696 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:58.812697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:58.812846 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:58.813405 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:58.813422 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:58.815101 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:58.817072 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:58.817078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:58.817081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:58.817358 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:58.817973 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:58.817973 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:58.817974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:58.818112 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:58.818479 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:58.818485 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:58.818643 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 17:47:58.819127 (+ 484us) client_negotiation.cc:240] Negotiation successful
0808 17:47:58.819286 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":83,"threads_started":1}
I20260808 17:47:58.820003 17720 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45003
I20260808 17:47:58.820101 17720 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:58.820278 17720 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:47:58.820621 16968 ts_manager.cc:194] Registered new tserver with Master: d2f42bb62118477faa85b87adfccd528 (127.29.62.195:36067)
I20260808 17:47:58.822125 16968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:59741
I20260808 17:47:58.828774 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:47:58.830685 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 17092
I20260808 17:47:58.844347 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 17313
I20260808 17:47:58.880280 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 17522
I20260808 17:47:58.893726 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 16914
I20260808 17:47:58.914079 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:45003
--webserver_interface=127.29.62.254
--webserver_port=45631
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:45003
--ranger_config_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:33487/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.62.212:33487/kms with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:47:58.997601 17725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:58.997807 17725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:58.997829 17725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:58.999018 17725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:47:58.999063 17725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:58.999078 17725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:58.999089 17725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:47:58.999101 17725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:47:59.000765 17725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.62.212:33487/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:45003
--ranger_config_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:45003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:59.001032 17725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:59.001260 17725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:59.006153 17735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:59.006130 17733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:59.006145 17737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:59.006408 17725 server_base.cc:1034] running on GCE node
I20260808 17:47:59.006659 17725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:59.007004 17725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:59.008170 17725 hybrid_clock.cc:648] HybridClock initialized: now 1786211279008161 us; error 39 us; skew 500 ppm
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:59.010783 17725 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:47:59.011216 17725 webserver.cc:460] Webserver started at http://127.29.62.254:45631/ using document root <none> and password file <none>
I20260808 17:47:59.011459 17725 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:59.011538 17725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:59.089401 17725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:47:59.090478 17747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.090703 17725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:47:59.090788 17725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "95ba15cdaa434cc4ad886ae4d3be6d10"
format_stamp: "Formatted at 2026-08-08 17:47:57 on dist-test-slave-tk9z"
server_key: "c6994dd4ecf12a7a90998b2503da2f78"
server_key_iv: "8fc6ace16961b6d6be16bcfc5a783674"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:59.091276 17725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:59.113015 17725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:59.114799 17725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:59.114965 17725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:59.119405 17725 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:45003
I20260808 17:47:59.119452 17826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:45003 every 8 connection(s)
I20260808 17:47:59.119831 17725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:47:59.121073 17828 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:47:59.121769 17828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Bootstrap starting.
I20260808 17:47:59.121968 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 17725
I20260808 17:47:59.122272 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:33487/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:46665
--local_ip_for_outbound_sockets=127.29.62.193
--tserver_master_addrs=127.29.62.254:45003
--webserver_port=42051
--webserver_interface=127.29.62.193
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.62.212:33487/kms with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:47:59.130196 17828 log.cc:824] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Log is configured to *not* fsync() on all Append() calls
I20260808 17:47:59.132099 17828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:47:59.132433 17828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Bootstrap complete.
I20260808 17:47:59.135074 17828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } }
I20260808 17:47:59.135368 17828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ba15cdaa434cc4ad886ae4d3be6d10, State: Initialized, Role: FOLLOWER
I20260808 17:47:59.135593 17828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } }
I20260808 17:47:59.135697 17828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:47:59.135754 17828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:47:59.135805 17828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:47:59.136865 17828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } }
I20260808 17:47:59.137002 17828 leader_election.cc:302] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95ba15cdaa434cc4ad886ae4d3be6d10; no voters:
I20260808 17:47:59.137240 17828 leader_election.cc:288] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:47:59.137280 17832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:47:59.137394 17832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [term 2 LEADER]: Becoming Leader. State: Replica: 95ba15cdaa434cc4ad886ae4d3be6d10, State: Running, Role: LEADER
I20260808 17:47:59.137547 17828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:47:59.137522 17832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } }
I20260808 17:47:59.137830 17832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95ba15cdaa434cc4ad886ae4d3be6d10. Latest consensus state: current_term: 2 leader_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } } }
I20260808 17:47:59.137914 17832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: This master's current role is: LEADER
I20260808 17:47:59.138079 17833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95ba15cdaa434cc4ad886ae4d3be6d10" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45003 } } }
I20260808 17:47:59.138140 17838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:47:59.138195 17833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10 [sys.catalog]: This master's current role is: LEADER
I20260808 17:47:59.138747 17838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:47:59.139230 17838 catalog_manager.cc:1259] Loaded cluster ID: 543c733d491644c0b41eca44c9e5edab
I20260808 17:47:59.139278 17838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:47:59.140074 17828 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 17:47:59.140480 17838 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:47:59.143950 17838 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Loaded TSK: 0
I20260808 17:47:59.144562 17838 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:47:59.245627 17830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:59.245929 17830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:59.245973 17830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:59.247776 17830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:59.247848 17830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:59.247895 17830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:47:59.250137 17830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.62.212:33487/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:46665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=42051
--tserver_master_addrs=127.29.62.254:45003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:59.250480 17830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:59.250766 17830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:59.257284 17889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:59.257366 17888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:59.257511 17891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:59.257652 17830 server_base.cc:1034] running on GCE node
I20260808 17:47:59.257862 17830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:59.258162 17830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:59.259351 17830 hybrid_clock.cc:648] HybridClock initialized: now 1786211279259319 us; error 45 us; skew 500 ppm
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:59.262773 17830 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:47:59.263445 17830 webserver.cc:460] Webserver started at http://127.29.62.193:42051/ using document root <none> and password file <none>
I20260808 17:47:59.263723 17830 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:59.263806 17830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:59.294821 17830 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.001s sys 0.000s
I20260808 17:47:59.299289 17908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.299490 17830 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260808 17:47:59.299584 17830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "da0078e49efd4405b0bc9604d23e433d"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "5ca02a38610d2f3d07a389888a3e24b0"
server_key_iv: "6ba8937d95886a74839c988f36b2170a"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:59.300143 17830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:59.322108 17830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:59.324735 17830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:59.324916 17830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:59.325153 17830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:59.325546 17830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:59.325594 17830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.325634 17830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:59.325659 17830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.333352 17830 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:46665
I20260808 17:47:59.333796 17830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:47:59.334523 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 17830
I20260808 17:47:59.334828 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:33487/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:46803
--local_ip_for_outbound_sockets=127.29.62.194
--tserver_master_addrs=127.29.62.254:45003
--webserver_port=41805
--webserver_interface=127.29.62.194
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.62.212:33487/kms with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:47:59.338694 18090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:46665 every 8 connection(s)
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:59.364785 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:59.334666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45003 (local address 127.29.62.193:53447)
0808 17:47:59.342617 (+ 7951us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:59.342680 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:59.343625 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:59.344434 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:59.344439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:59.344718 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:59.345047 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:59.345055 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.346960 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.346962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:59.348264 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:59.348267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.349245 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.349246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:59.349375 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:59.350101 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:59.350112 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:59.360338 (+ 10226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:59.362892 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:59.362919 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:59.362921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:59.363232 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:59.363520 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:59.363520 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:59.363521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:59.363647 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:59.364128 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:59.364134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:59.364282 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 17:47:59.364384 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 17:47:59.364553 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7893,"thread_start_us":89,"threads_started":1}
I20260808 17:47:59.365327 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:59.335071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:53447 (local address 127.29.62.254:45003)
0808 17:47:59.335290 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:59.335300 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:59.342860 (+ 7560us) server_negotiation.cc:407] Connection header received
0808 17:47:59.343965 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:59.343976 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:59.344056 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:59.344184 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:59.345207 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.346802 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.348371 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.349022 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.349103 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:59.360532 (+ 11429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:59.360540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:59.360541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:59.360556 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:59.362780 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:59.363329 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:59.363333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:59.363333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:59.363445 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:59.363714 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:59.363715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:59.363715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:59.364022 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:59.364098 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:59.365009 (+ 911us) server_negotiation.cc:299] Negotiation successful
0808 17:47:59.365139 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":86,"threads_started":1}
I20260808 17:47:59.366024 18091 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45003
I20260808 17:47:59.366134 18091 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.366330 18091 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
W20260808 17:47:59.367273 18091 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:45003 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 17:47:59.367357 18091 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.367440 18091 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:47:59.367774 18091 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.367861 18091 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
W20260808 17:47:59.368227 18091 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:47:59.459363 18095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:59.459671 18095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:59.459719 18095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:59.461427 18095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:59.461490 18095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:59.461536 18095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:47:59.463346 18095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.62.212:33487/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:46803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=41805
--tserver_master_addrs=127.29.62.254:45003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:59.463631 18095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:59.463899 18095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:59.473649 18106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:59.473692 18107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:59.473714 18109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:59.474682 18095 server_base.cc:1034] running on GCE node
I20260808 17:47:59.474969 18095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:59.475266 18095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:59.494343 18095 hybrid_clock.cc:648] HybridClock initialized: now 1786211279494306 us; error 49 us; skew 500 ppm
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:59.497895 18095 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:47:59.498575 18095 webserver.cc:460] Webserver started at http://127.29.62.194:41805/ using document root <none> and password file <none>
I20260808 17:47:59.498924 18095 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:59.499068 18095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:59.526568 18095 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 17:47:59.527997 18122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.528241 18095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:47:59.528342 18095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "e8cc232cc5484e159a21c4661ba2177e"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "3f9dc976c12cc99ad4844a2bb9ae1b8a"
server_key_iv: "eed6dc71ee3d125036c4d6521cb046c6"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:59.528833 18095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:59.541595 18095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:59.543829 18095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:59.543977 18095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:59.544210 18095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:59.544540 18095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:59.544584 18095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.544617 18095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:59.544638 18095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.551743 18095 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:46803
I20260808 17:47:59.552189 18095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:47:59.552587 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 18095
I20260808 17:47:59.552929 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.62.212:33487/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:36067
--local_ip_for_outbound_sockets=127.29.62.195
--tserver_master_addrs=127.29.62.254:45003
--webserver_port=36695
--webserver_interface=127.29.62.195
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.62.212:33487/kms with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:47:59.554759 18299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:46803 every 8 connection(s)
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:59.566226 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:59.554489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:54305 (local address 127.29.62.254:45003)
0808 17:47:59.554539 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:59.554542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:59.555071 (+ 529us) server_negotiation.cc:407] Connection header received
0808 17:47:59.555868 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:59.555871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:59.555884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:59.555918 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:59.556780 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.557562 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.559240 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.559599 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.559649 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:59.562527 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:59.562529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:59.562529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:59.562538 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:59.564377 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:59.564940 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:59.564942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:59.564943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:59.565006 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:59.565346 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:59.565347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:59.565347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:59.565512 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:59.565576 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:59.565986 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 17:47:59.566086 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:47:59.566869 18302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:59.554157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45003 (local address 127.29.62.194:54305)
0808 17:47:59.554931 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:59.554976 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:59.555730 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:59.556112 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:59.556117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:59.556375 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:59.556668 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:59.556675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.557692 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.557693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:59.559122 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:59.559125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.559773 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.559775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:59.559924 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:59.560531 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:59.560544 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:59.562352 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:59.564493 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:59.564501 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:59.564504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:59.564840 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:59.565085 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:59.565086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:59.565087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:59.565255 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:59.565615 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:59.565622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:59.565805 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 17:47:59.566419 (+ 614us) client_negotiation.cc:240] Negotiation successful
0808 17:47:59.566608 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":92,"threads_started":1}
I20260808 17:47:59.567299 18300 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45003
I20260808 17:47:59.567394 18300 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.567595 18300 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
W20260808 17:47:59.568192 18300 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:45003 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 17:47:59.568267 18300 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.568334 18300 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:47:59.568677 18300 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.568748 18300 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
W20260808 17:47:59.568991 18300 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:47:59.675889 18304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:47:59.676203 18304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:47:59.676249 18304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:47:59.677709 18304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:47:59.677765 18304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:47:59.677806 18304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:47:59.680162 18304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.62.212:33487/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:36067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=36695
--tserver_master_addrs=127.29.62.254:45003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:47:59.680436 18304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:47:59.680707 18304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:47:59.686839 18312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:59.687170 18304 server_base.cc:1034] running on GCE node
W20260808 17:47:59.687116 18311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:47:59.687388 18314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:47:59.687642 18304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:47:59.687927 18304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:47:59.689085 18304 hybrid_clock.cc:648] HybridClock initialized: now 1786211279689079 us; error 33 us; skew 500 ppm
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:47:59.699425 18304 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:47:59.700035 18304 webserver.cc:460] Webserver started at http://127.29.62.195:36695/ using document root <none> and password file <none>
I20260808 17:47:59.700388 18304 fs_manager.cc:359] Metadata directory not provided
I20260808 17:47:59.700528 18304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:47:59.717657 18304 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:47:59.718822 18327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.719084 18304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:47:59.719197 18304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "d2f42bb62118477faa85b87adfccd528"
format_stamp: "Formatted at 2026-08-08 17:47:58 on dist-test-slave-tk9z"
server_key: "884fa11d4357279db009cbeb25ee9ed9"
server_key_iv: "c960367faf786d03757050c7484bf565"
server_key_version: "kuduclusterkey@0"
I20260808 17:47:59.719687 18304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:47:59.753543 18304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:47:59.756045 18304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:47:59.756206 18304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:47:59.756460 18304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:47:59.756865 18304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:47:59.756933 18304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.756980 18304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:47:59.757009 18304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:47:59.767402 18304 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:36067
I20260808 17:47:59.768081 18304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:47:59.768113 18518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:36067 every 8 connection(s)
I20260808 17:47:59.773169 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 18304
Aug 08 17:47:59 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211279, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:47:59.780423 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:59.769114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:43431 (local address 127.29.62.254:45003)
0808 17:47:59.769185 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:59.769187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:59.769443 (+ 256us) server_negotiation.cc:407] Connection header received
0808 17:47:59.770131 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:59.770133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:59.770147 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:59.770210 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:59.771031 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.771766 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.773320 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.773635 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.773682 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:59.777138 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:59.777140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:59.777141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:59.777149 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:59.778872 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:59.779379 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:59.779380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:59.779380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:59.779437 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:59.779695 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:59.779696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:59.779696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:59.779817 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:59.779855 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:59.780233 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 17:47:59.780318 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:47:59.781215 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:59.769098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45003 (local address 127.29.62.195:43431)
0808 17:47:59.769275 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 17:47:59.769325 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 17:47:59.769997 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:47:59.770324 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:47:59.770329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:47:59.770609 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:47:59.770902 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:59.770910 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.771890 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.771892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:59.773206 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:47:59.773209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.774014 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.774015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:47:59.774170 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:59.775023 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:47:59.775034 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:47:59.777015 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:47:59.778978 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:59.778985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:59.778988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:59.779293 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:59.779506 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:47:59.779506 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:47:59.779507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:47:59.779629 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:47:59.779896 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:47:59.779901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:47:59.780082 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 17:47:59.780796 (+ 714us) client_negotiation.cc:240] Negotiation successful
0808 17:47:59.780976 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260808 17:47:59.781840 18519 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45003
I20260808 17:47:59.781944 18519 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.782123 18519 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
W20260808 17:47:59.782775 18519 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:45003 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 17:47:59.782840 18519 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.782894 18519 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:47:59.783022 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:47:59.773810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53624 (local address 127.29.62.254:45003)
0808 17:47:59.774005 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 17:47:59.774007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:47:59.774014 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:47:59.774036 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:47:59.774037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:47:59.774049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:47:59.775271 (+ 1222us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:47:59.775308 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.776131 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.777675 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:47:59.778230 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:47:59.779737 (+ 1507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:47:59.779791 (+ 54us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:47:59.779792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:47:59.779792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:47:59.779797 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:47:59.781460 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:59.781862 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:59.781863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:59.781864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:59.781917 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:47:59.782106 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:47:59.782106 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:47:59.782107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:47:59.782216 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:47:59.782798 (+ 582us) server_negotiation.cc:1035] Waiting for connection context
0808 17:47:59.782839 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 17:47:59.782920 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
I20260808 17:47:59.783710 18519 heartbeater.cc:464] Registering TS with master...
I20260808 17:47:59.783787 18519 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
W20260808 17:47:59.784051 18519 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 17:48:00 dist-test-slave-tk9z krb5kdc[16184](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786211280, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:00.368417 18091 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:00.368602 18091 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:48:00.369041 17777 ts_manager.cc:194] Registered new tserver with Master: da0078e49efd4405b0bc9604d23e433d (127.29.62.193:46665)
I20260808 17:48:00.370059 17777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:53447
I20260808 17:48:00.569249 18300 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:00.569453 18300 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:48:00.569856 17777 ts_manager.cc:194] Registered new tserver with Master: e8cc232cc5484e159a21c4661ba2177e (127.29.62.194:46803)
I20260808 17:48:00.570485 17777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:54305
Aug 08 17:48:00 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211280, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
I20260808 17:48:00.784245 18519 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:00.784471 18519 heartbeater.cc:511] Master 127.29.62.254:45003 requested a full tablet report, sending...
I20260808 17:48:00.784927 17777 ts_manager.cc:194] Registered new tserver with Master: d2f42bb62118477faa85b87adfccd528 (127.29.62.195:36067)
I20260808 17:48:00.785467 17777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:43431
I20260808 17:48:00.791075 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:48:00.800927 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:00.792205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53634 (local address 127.29.62.254:45003)
0808 17:48:00.792540 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:00.792542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:00.792549 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:48:00.792573 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:00.792574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:00.792585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:00.792643 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:00.793430 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.794286 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.795402 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.795691 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.795761 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:00.797361 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:00.797363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:00.797363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:00.797370 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:00.799293 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:00.799720 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:00.799721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:00.799722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:00.799786 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:00.800015 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:00.800016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:00.800016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:00.800148 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:00.800235 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:00.800723 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 17:48:00.800808 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":62,"threads_started":1}
I20260808 17:48:00.803463 17777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:48:00.806396 17777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:48:00.819342 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:00.813014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:46803 (local address 127.0.0.1:41812)
0808 17:48:00.814504 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:00.814547 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:00.814614 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:00.815190 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:00.815194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:00.815196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:00.815269 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:00.815272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.816669 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.816670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:00.817871 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:00.817875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.819032 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.819033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:00.819112 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:00.819126 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 17:48:00.819195 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 17:48:00.819250 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1453,"thread_start_us":131,"threads_started":1}
I20260808 17:48:00.819720 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:00.813848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.29.62.194:46803)
0808 17:48:00.814902 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:00.814907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:00.814918 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:48:00.814953 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:00.814955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:00.814971 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:00.815038 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:00.815536 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.816568 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.817966 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.818838 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.818925 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:00.818991 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:00.819599 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 17:48:00.819643 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":55,"threads_started":1}
I20260808 17:48:00.820458 18194 tablet_service.cc:1467] Processing CreateTablet for tablet 2119e67503784a54a89632f48ee63bce (DEFAULT_TABLE table=test-table [id=a411bcd688224b8e9d3fc2a3443088b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:00.820897 18194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2119e67503784a54a89632f48ee63bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:00.826853 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:00.813285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:36067 (local address 127.0.0.1:53582)
0808 17:48:00.814758 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:00.814767 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:00.814803 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:00.822722 (+ 7919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:00.822724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:00.822727 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:00.822784 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:00.822786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.824202 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.824204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:00.825423 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:00.825427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.826611 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.826612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:00.826683 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:00.826689 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:48:00.826718 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 17:48:00.826764 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1414,"thread_start_us":105,"threads_started":1}
I20260808 17:48:00.827323 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:00.814001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53582 (local address 127.29.62.195:36067)
0808 17:48:00.822479 (+ 8478us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:00.822485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:00.822500 (+ 15us) server_negotiation.cc:407] Connection header received
0808 17:48:00.822542 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:00.822544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:00.822562 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:00.822857 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:00.822967 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.824100 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.825527 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.826505 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.827056 (+ 551us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:00.827133 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:00.827160 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 17:48:00.827200 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8431,"thread_start_us":51,"threads_started":1}
I20260808 17:48:00.828078 18406 tablet_service.cc:1467] Processing CreateTablet for tablet 2119e67503784a54a89632f48ee63bce (DEFAULT_TABLE table=test-table [id=a411bcd688224b8e9d3fc2a3443088b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:00.828529 18406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2119e67503784a54a89632f48ee63bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:00.833287 18572 tablet_bootstrap.cc:492] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Bootstrap starting.
I20260808 17:48:00.833323 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:00.813483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:46665 (local address 127.0.0.1:56560)
0808 17:48:00.815408 (+ 1925us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:00.815417 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:00.815466 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:00.826820 (+ 11354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:00.826821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:00.826825 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:00.826875 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:00.826877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.828172 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.828173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:00.829326 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:00.829330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.833080 (+ 3750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.833081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:00.833151 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:00.833158 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:48:00.833190 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 17:48:00.833236 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1904,"thread_start_us":70,"threads_started":1}
I20260808 17:48:00.833709 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:00.814137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56560 (local address 127.29.62.193:46665)
0808 17:48:00.826534 (+ 12397us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:00.826539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:00.826553 (+ 14us) server_negotiation.cc:407] Connection header received
0808 17:48:00.826592 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:00.826594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:00.826614 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:00.826708 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:00.826972 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.828063 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.829451 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:00.830316 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:00.833464 (+ 3148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:00.833544 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:00.833576 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 17:48:00.833621 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12351,"thread_start_us":54,"threads_started":1}
I20260808 17:48:00.834301 18572 tablet_bootstrap.cc:654] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:00.834487 17989 tablet_service.cc:1467] Processing CreateTablet for tablet 2119e67503784a54a89632f48ee63bce (DEFAULT_TABLE table=test-table [id=a411bcd688224b8e9d3fc2a3443088b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:00.834786 18572 log.cc:824] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:00.834874 17989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2119e67503784a54a89632f48ee63bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:00.835592 18572 tablet_bootstrap.cc:492] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: No bootstrap required, opened a new log
I20260808 17:48:00.835664 18572 ts_tablet_manager.cc:1397] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:48:00.838961 18572 raft_consensus.cc:348] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:00.839144 18572 raft_consensus.cc:374] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:00.839180 18572 raft_consensus.cc:729] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f42bb62118477faa85b87adfccd528, State: Initialized, Role: FOLLOWER
I20260808 17:48:00.839313 18572 consensus_queue.cc:260] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:00.839581 18572 ts_tablet_manager.cc:1428] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 17:48:00.839705 18519 heartbeater.cc:503] Master 127.29.62.254:45003 was elected leader, sending a full tablet report...
I20260808 17:48:00.840893 18574 tablet_bootstrap.cc:492] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d: Bootstrap starting.
I20260808 17:48:00.841969 18574 tablet_bootstrap.cc:654] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:00.842494 18574 log.cc:824] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:00.846686 18571 tablet_bootstrap.cc:492] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Bootstrap starting.
I20260808 17:48:00.847222 18574 tablet_bootstrap.cc:492] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d: No bootstrap required, opened a new log
I20260808 17:48:00.847303 18574 ts_tablet_manager.cc:1397] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260808 17:48:00.847833 18571 tablet_bootstrap.cc:654] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:00.848645 18571 log.cc:824] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:00.851109 18574 raft_consensus.cc:348] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:00.851302 18574 raft_consensus.cc:374] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:00.851341 18574 raft_consensus.cc:729] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da0078e49efd4405b0bc9604d23e433d, State: Initialized, Role: FOLLOWER
I20260808 17:48:00.851487 18574 consensus_queue.cc:260] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:00.851810 18574 ts_tablet_manager.cc:1428] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260808 17:48:00.851945 18091 heartbeater.cc:503] Master 127.29.62.254:45003 was elected leader, sending a full tablet report...
I20260808 17:48:00.853765 18571 tablet_bootstrap.cc:492] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: No bootstrap required, opened a new log
I20260808 17:48:00.853845 18571 ts_tablet_manager.cc:1397] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260808 17:48:00.856482 18571 raft_consensus.cc:348] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:00.856660 18571 raft_consensus.cc:374] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:00.856698 18571 raft_consensus.cc:729] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8cc232cc5484e159a21c4661ba2177e, State: Initialized, Role: FOLLOWER
I20260808 17:48:00.856832 18571 consensus_queue.cc:260] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:00.857144 18571 ts_tablet_manager.cc:1428] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 17:48:00.857264 18300 heartbeater.cc:503] Master 127.29.62.254:45003 was elected leader, sending a full tablet report...
Aug 08 17:48:01 dist-test-slave-tk9z krb5kdc[16184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211280, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for HTTP/127.29.62.212@KRBTEST.COM
W20260808 17:48:01.019974 18520 tablet.cc:2369] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:48:01.056361 18301 tablet.cc:2369] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:48:01.090517 18092 tablet.cc:2369] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:48:01.303210 18575 raft_consensus.cc:482] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:01.303395 18575 raft_consensus.cc:504] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:01.303977 18575 leader_election.cc:288] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8cc232cc5484e159a21c4661ba2177e (127.29.62.194:46803), da0078e49efd4405b0bc9604d23e433d (127.29.62.193:46665)
I20260808 17:48:01.308171 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:01.304528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:56693 (local address 127.29.62.193:46665)
0808 17:48:01.304604 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:01.304607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:01.304676 (+ 69us) server_negotiation.cc:407] Connection header received
0808 17:48:01.304778 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:01.304780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:01.304790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:01.304847 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:01.305165 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.305983 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.307372 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.307785 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.307878 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:01.307889 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:01.308058 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 17:48:01.308101 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:48:01.308190 18598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:01.304379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:46665 (local address 127.29.62.195:56693)
0808 17:48:01.304605 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:01.304616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:01.304685 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:01.304857 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:01.304858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:01.304859 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:01.305066 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:01.305068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.306083 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.306084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:01.307263 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:01.307264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.307907 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.307909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:01.307984 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:01.307987 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:48:01.308052 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:48:01.308091 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":8,"thread_start_us":58,"threads_started":1}
I20260808 17:48:01.308555 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:01.304346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:43871 (local address 127.29.62.194:46803)
0808 17:48:01.304460 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:01.304463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:01.304626 (+ 163us) server_negotiation.cc:407] Connection header received
0808 17:48:01.304754 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:01.304755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:01.304766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:01.304799 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:01.305059 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.305685 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.307334 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.308032 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.308117 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:01.308125 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:01.308446 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 17:48:01.308495 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 17:48:01.308625 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:01.304179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:46803 (local address 127.29.62.195:43871)
0808 17:48:01.304510 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:01.304524 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:01.304664 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:01.304882 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:01.304883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:01.304885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:01.304965 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:01.304967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.305777 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.305778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:01.307229 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:01.307232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.308165 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.308166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:01.308305 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:01.308309 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:01.308408 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 17:48:01.308451 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":86,"threads_started":1}
I20260808 17:48:01.308872 18016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2119e67503784a54a89632f48ee63bce" candidate_uuid: "d2f42bb62118477faa85b87adfccd528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da0078e49efd4405b0bc9604d23e433d" is_pre_election: true
I20260808 17:48:01.309031 18016 raft_consensus.cc:2454] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f42bb62118477faa85b87adfccd528 in term 0.
I20260808 17:48:01.309154 18233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2119e67503784a54a89632f48ee63bce" candidate_uuid: "d2f42bb62118477faa85b87adfccd528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8cc232cc5484e159a21c4661ba2177e" is_pre_election: true
I20260808 17:48:01.309317 18233 raft_consensus.cc:2454] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f42bb62118477faa85b87adfccd528 in term 0.
I20260808 17:48:01.309288 18329 leader_election.cc:302] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2f42bb62118477faa85b87adfccd528, da0078e49efd4405b0bc9604d23e433d; no voters:
I20260808 17:48:01.309432 18575 raft_consensus.cc:2781] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:48:01.309486 18575 raft_consensus.cc:482] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:48:01.309510 18575 raft_consensus.cc:3037] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:01.310411 18575 raft_consensus.cc:504] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:01.310557 18575 leader_election.cc:288] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [CANDIDATE]: Term 1 election: Requested vote from peers e8cc232cc5484e159a21c4661ba2177e (127.29.62.194:46803), da0078e49efd4405b0bc9604d23e433d (127.29.62.193:46665)
I20260808 17:48:01.310738 18016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2119e67503784a54a89632f48ee63bce" candidate_uuid: "d2f42bb62118477faa85b87adfccd528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da0078e49efd4405b0bc9604d23e433d"
I20260808 17:48:01.310842 18016 raft_consensus.cc:3037] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:01.310810 18233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2119e67503784a54a89632f48ee63bce" candidate_uuid: "d2f42bb62118477faa85b87adfccd528" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8cc232cc5484e159a21c4661ba2177e"
I20260808 17:48:01.310921 18233 raft_consensus.cc:3037] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:01.311782 18016 raft_consensus.cc:2454] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f42bb62118477faa85b87adfccd528 in term 1.
I20260808 17:48:01.311827 18233 raft_consensus.cc:2454] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f42bb62118477faa85b87adfccd528 in term 1.
I20260808 17:48:01.312023 18329 leader_election.cc:302] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2f42bb62118477faa85b87adfccd528, da0078e49efd4405b0bc9604d23e433d; no voters:
I20260808 17:48:01.312134 18575 raft_consensus.cc:2781] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:01.312340 18575 raft_consensus.cc:686] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 1 LEADER]: Becoming Leader. State: Replica: d2f42bb62118477faa85b87adfccd528, State: Running, Role: LEADER
I20260808 17:48:01.312476 18575 consensus_queue.cc:237] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } }
I20260808 17:48:01.313448 17777 catalog_manager.cc:5495] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2f42bb62118477faa85b87adfccd528 (127.29.62.195). New cstate: current_term: 1 leader_uuid: "d2f42bb62118477faa85b87adfccd528" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2f42bb62118477faa85b87adfccd528" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 36067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 } health_report { overall_health: UNKNOWN } } }
I20260808 17:48:01.365340 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:01.362874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.29.62.195:36067)
0808 17:48:01.362988 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:01.362990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:01.363125 (+ 135us) server_negotiation.cc:407] Connection header received
0808 17:48:01.363184 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:01.363185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:01.363205 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:01.363280 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:48:01.363408 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.363881 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.364614 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:01.364813 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:01.364870 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:01.365019 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:48:01.365118 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:48:01.365180 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:01.365233 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 17:48:01.365250 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
I20260808 17:48:01.367375 18016 raft_consensus.cc:1261] T 2119e67503784a54a89632f48ee63bce P da0078e49efd4405b0bc9604d23e433d [term 1 FOLLOWER]: Refusing update from remote peer d2f42bb62118477faa85b87adfccd528: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:01.367434 18233 raft_consensus.cc:1261] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 1 FOLLOWER]: Refusing update from remote peer d2f42bb62118477faa85b87adfccd528: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:01.367920 18601 consensus_queue.cc:1035] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8cc232cc5484e159a21c4661ba2177e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 46803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:01.368062 18575 consensus_queue.cc:1035] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da0078e49efd4405b0bc9604d23e433d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 46665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:01.372169 18615 mvcc.cc:205] Tried to move back new op lower bound from 7316321408478142464 to 7316321408256765952. Current Snapshot: MvccSnapshot[applied={T|T < 7316321408478142464}]
I20260808 17:48:01.373497 17776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:01.373615 17776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:01.374440 17776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95ba15cdaa434cc4ad886ae4d3be6d10: Sending DeleteTablet for 3 replicas of tablet 2119e67503784a54a89632f48ee63bce
I20260808 17:48:01.374760 18194 tablet_service.cc:1514] Processing DeleteTablet for tablet 2119e67503784a54a89632f48ee63bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:01 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:41812
I20260808 17:48:01.374843 18406 tablet_service.cc:1514] Processing DeleteTablet for tablet 2119e67503784a54a89632f48ee63bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:01 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:53582
I20260808 17:48:01.375056 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 17830
I20260808 17:48:01.375037 17989 tablet_service.cc:1514] Processing DeleteTablet for tablet 2119e67503784a54a89632f48ee63bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:01 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:56560
I20260808 17:48:01.375118 18627 tablet_replica.cc:330] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: stopping tablet replica
I20260808 17:48:01.375207 18627 raft_consensus.cc:2229] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:01.375291 18627 raft_consensus.cc:2258] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:01.375640 18627 ts_tablet_manager.cc:1905] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:01.377408 18628 tablet_replica.cc:330] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: stopping tablet replica
I20260808 17:48:01.377508 18628 raft_consensus.cc:2229] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:48:01.377599 18628 raft_consensus.cc:2258] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:01.379402 18628 ts_tablet_manager.cc:1905] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:01.381009 18628 ts_tablet_manager.cc:1918] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:01.381101 18628 log.cc:1196] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/2119e67503784a54a89632f48ee63bce
I20260808 17:48:01.381353 18628 ts_tablet_manager.cc:1939] T 2119e67503784a54a89632f48ee63bce P d2f42bb62118477faa85b87adfccd528: Deleting consensus metadata
I20260808 17:48:01.381963 17749 catalog_manager.cc:4841] TS d2f42bb62118477faa85b87adfccd528 (127.29.62.195:36067): tablet 2119e67503784a54a89632f48ee63bce (table test-table [id=a411bcd688224b8e9d3fc2a3443088b8]) successfully deleted
I20260808 17:48:01.383244 18627 ts_tablet_manager.cc:1918] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:01.383350 18627 log.cc:1196] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/2119e67503784a54a89632f48ee63bce
I20260808 17:48:01.383639 18627 ts_tablet_manager.cc:1939] T 2119e67503784a54a89632f48ee63bce P e8cc232cc5484e159a21c4661ba2177e: Deleting consensus metadata
I20260808 17:48:01.384121 17749 catalog_manager.cc:4841] TS e8cc232cc5484e159a21c4661ba2177e (127.29.62.194:46803): tablet 2119e67503784a54a89632f48ee63bce (table test-table [id=a411bcd688224b8e9d3fc2a3443088b8]) successfully deleted
W20260808 17:48:01.387290 17749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:46665 (error 108)
I20260808 17:48:01.387705 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 18095
I20260808 17:48:01.387861 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:01.387676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:46665 (local address 127.0.0.1:56564)
0808 17:48:01.387774 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:01.387815 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:46665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":59,"threads_started":1}
W20260808 17:48:01.394953 17749 catalog_manager.cc:4584] TS da0078e49efd4405b0bc9604d23e433d (127.29.62.193:46665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2119e67503784a54a89632f48ee63bce: Network error: Client connection negotiation failed: client connection to 127.29.62.193:46665: connect: Connection refused (error 111)
I20260808 17:48:01.408380 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 18304
I20260808 17:48:01.416263 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 17725
I20260808 17:48:01.429383 29947 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 17:48:01.775825 29947 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 17:48:01.775918 29947 mini_ranger.cc:66] Stopping Ranger...
I20260808 17:48:02.130498 29947 mini_ranger.cc:68] Stopped Ranger
2026-08-08 17:48:02.130 UTC [16233] LOG: received smart shutdown request
2026-08-08 17:48:02.134 UTC [16233] LOG: background worker "logical replication launcher" (PID 16248) exited with exit code 1
2026-08-08 17:48:02.135 UTC [16240] LOG: shutting down
2026-08-08 17:48:02.269 UTC [16233] LOG: database system is shut down
2026-08-08T17:48:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49629 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:48:02 dist-test-slave-tk9z krb5kdc[18646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:48:02 dist-test-slave-tk9z krb5kdc[18646](info): set up 2 sockets
Aug 08 17:48:02 dist-test-slave-tk9z krb5kdc[18646](info): commencing operation
krb5kdc: starting...
W20260808 17:48:03.643535 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:48:03 dist-test-slave-tk9z krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:48:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:48:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:48:03.768991 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:33565
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:33565,127.29.62.253:33073,127.29.62.252:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:03.862248 18668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:03.862493 18668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:03.862519 18668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:03.863785 18668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:03.863831 18668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:03.863848 18668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:03.863910 18668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:03.863931 18668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:03.865635 18668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:33565,127.29.62.253:33073,127.29.62.252:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:33565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:03.865917 18668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:03.866187 18668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:03.871217 18680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:03.871431 18668 server_base.cc:1034] running on GCE node
W20260808 17:48:03.871206 18674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:03.871217 18675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:03.871935 18668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:03.872265 18668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:03.873411 18668 hybrid_clock.cc:648] HybridClock initialized: now 1786211283873395 us; error 29 us; skew 500 ppm
Aug 08 17:48:03 dist-test-slave-tk9z krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211283, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:03.876044 18668 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:48:03.876482 18668 webserver.cc:460] Webserver started at http://127.29.62.254:32845/ using document root <none> and password file <none>
I20260808 17:48:03.876734 18668 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:03.876807 18668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:03.876956 18668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:03.878063 18668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "3766e756f78c45f08194a232fd94f34f"
format_stamp: "Formatted at 2026-08-08 17:48:03 on dist-test-slave-tk9z"
server_key: "9adb7447e7910cccf003e29cf9bbe716"
server_key_iv: "94d63df8f756775b079c7481871cf9e4"
server_key_version: "encryptionkey@0"
I20260808 17:48:03.878510 18668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "3766e756f78c45f08194a232fd94f34f"
format_stamp: "Formatted at 2026-08-08 17:48:03 on dist-test-slave-tk9z"
server_key: "9adb7447e7910cccf003e29cf9bbe716"
server_key_iv: "94d63df8f756775b079c7481871cf9e4"
server_key_version: "encryptionkey@0"
I20260808 17:48:03.879886 18668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:03.880630 18694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:03.880822 18668 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:48:03.880930 18668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "3766e756f78c45f08194a232fd94f34f"
format_stamp: "Formatted at 2026-08-08 17:48:03 on dist-test-slave-tk9z"
server_key: "9adb7447e7910cccf003e29cf9bbe716"
server_key_iv: "94d63df8f756775b079c7481871cf9e4"
server_key_version: "encryptionkey@0"
I20260808 17:48:03.881021 18668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:03.891023 18668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:03.893224 18668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:03.893406 18668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:03.897724 18668 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:33565
I20260808 17:48:03.897761 18780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:33565 every 8 connection(s)
I20260808 17:48:03.898168 18668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:48:03.899217 18781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:03.900960 18781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:48:03.904932 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 18668
I20260808 17:48:03.905033 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:48:03.905228 29947 external_mini_cluster.cc:1428] Setting key b0f15e6dcdbb26e6da29c8b6d391cd3c
Aug 08 17:48:03 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211283, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:03.912139 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:03.901696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:33565 (local address 127.0.0.1:53820)
0808 17:48:03.901885 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:03.901923 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:03.902470 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:03.902812 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:03.902815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:03.903080 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:03.903246 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:03.903257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:03.904676 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:03.904677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:03.905509 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:03.905513 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:03.906030 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:03.906031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:03.906097 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:03.906753 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:03.906762 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:03.908451 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:03.910656 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:03.910663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:03.910665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:03.910863 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:03.911154 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:03.911155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:03.911155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:03.911237 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:03.911700 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:03.911705 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:03.911760 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 17:48:03.911832 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:48:03.911957 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260808 17:48:03.912235 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:03.901872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.29.62.254:33565)
0808 17:48:03.902015 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:03.902019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:03.902060 (+ 41us) server_negotiation.cc:407] Connection header received
0808 17:48:03.902633 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:03.902636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:03.902659 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:03.902761 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:03.903356 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:03.904566 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:03.905610 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:03.905928 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:03.906059 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:03.908533 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:03.908541 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:03.908542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:03.908550 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:03.910573 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:03.910953 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:03.910957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:03.910957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:03.911077 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:03.911306 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:03.911307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:03.911307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:03.911622 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:03.911721 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:03.911903 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 17:48:03.912050 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260808 17:48:03.913009 18781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:48:03.913390 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:03.913262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38252)
0808 17:48:03.913316 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:03.913369 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 17:48:03.913561 18703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
I20260808 17:48:03.913991 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:03.913892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38266)
0808 17:48:03.913937 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:03.913962 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:48:03.914143 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:33073: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.62.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab.
Entry for principal kudu/127.29.62.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab.
I20260808 17:48:03.931496 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.253:33073
--webserver_interface=127.29.62.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab
--principal=kudu/127.29.62.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:33565,127.29.62.253:33073,127.29.62.252:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:03.963385 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } attempt: 1
I20260808 17:48:03.963886 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:03.963742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38282)
0808 17:48:03.963815 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:03.963853 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 17:48:03.964267 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:03.964165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38286)
0808 17:48:03.964214 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:03.964247 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 17:48:03.964404 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:33073: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
W20260808 17:48:04.016705 18792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:04.016920 18792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:04.016943 18792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:04.018155 18792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:04.018208 18792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:04.018222 18792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:04.018271 18792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:04.018301 18792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:04.019893 18792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:33565,127.29.62.253:33073,127.29.62.252:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab
--principal=kudu/127.29.62.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.253:33073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.62.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:04.020154 18792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:04.020395 18792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:04.025254 18799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:04.025293 18792 server_base.cc:1034] running on GCE node
W20260808 17:48:04.025236 18801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:04.025228 18803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:04.025868 18792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:04.026221 18792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:04.027391 18792 hybrid_clock.cc:648] HybridClock initialized: now 1786211284027380 us; error 46 us; skew 500 ppm
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211284, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:04.030102 18792 init.cc:377] Logged in from keytab as kudu/127.29.62.253@KRBTEST.COM (short username kudu)
I20260808 17:48:04.030635 18792 webserver.cc:460] Webserver started at http://127.29.62.253:34407/ using document root <none> and password file <none>
I20260808 17:48:04.030872 18792 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:04.030942 18792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:04.031105 18792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:04.032214 18792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/data/instance:
uuid: "9663937117f94649a80b50ba836bc78c"
format_stamp: "Formatted at 2026-08-08 17:48:04 on dist-test-slave-tk9z"
server_key: "677b989ae6947c5552307fe2909fb22a"
server_key_iv: "0660e7f1f7daaabd591132f91eae8f31"
server_key_version: "encryptionkey@0"
I20260808 17:48:04.032563 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } attempt: 2
I20260808 17:48:04.032622 18792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/wal/instance:
uuid: "9663937117f94649a80b50ba836bc78c"
format_stamp: "Formatted at 2026-08-08 17:48:04 on dist-test-slave-tk9z"
server_key: "677b989ae6947c5552307fe2909fb22a"
server_key_iv: "0660e7f1f7daaabd591132f91eae8f31"
server_key_version: "encryptionkey@0"
I20260808 17:48:04.033004 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.032863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38302)
0808 17:48:04.032933 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.032966 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 17:48:04.033432 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.033289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38318)
0808 17:48:04.033358 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.033395 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 17:48:04.033569 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:33073: Network error: Client connection negotiation failed: client connection to 127.29.62.253:33073: connect: Connection refused (error 111)
I20260808 17:48:04.034296 18792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:48:04.035127 18814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:04.035326 18792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:04.035408 18792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/wal
uuid: "9663937117f94649a80b50ba836bc78c"
format_stamp: "Formatted at 2026-08-08 17:48:04 on dist-test-slave-tk9z"
server_key: "677b989ae6947c5552307fe2909fb22a"
server_key_iv: "0660e7f1f7daaabd591132f91eae8f31"
server_key_version: "encryptionkey@0"
I20260808 17:48:04.035506 18792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:04.056834 18792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:04.059404 18792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:04.059564 18792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:04.063670 18792 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.253:33073
I20260808 17:48:04.063753 18890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.253:33073 every 8 connection(s)
I20260808 17:48:04.064109 18792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
I20260808 17:48:04.065071 18891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:04.066856 18891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:48:04.067977 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 18792
I20260808 17:48:04.068099 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-1/wal/instance
I20260808 17:48:04.068262 29947 external_mini_cluster.cc:1428] Setting key 4d51b2b0ccbe567f781a55c8bab59800
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211284, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:04.077006 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.067648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53828 (local address 127.29.62.254:33565)
0808 17:48:04.067710 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.067712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.067901 (+ 189us) server_negotiation.cc:407] Connection header received
0808 17:48:04.068550 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.068551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.068568 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.068620 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.069258 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.069766 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.070878 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.071101 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.071192 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.073846 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.073848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.073849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.073857 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.075617 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.076013 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.076014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.076015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.076088 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.076324 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.076325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.076326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.076483 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.076565 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.076818 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.076909 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:48:04.077212 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.067604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:33565 (local address 127.0.0.1:53828)
0808 17:48:04.067778 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.067817 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:04.068422 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:04.068688 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:04.068693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:04.068948 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:04.069156 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.069160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.069870 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.069871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.070781 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.070784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.071204 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.071205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.071361 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.071909 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:04.071920 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:04.073744 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:04.075711 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.075717 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.075719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.075932 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.076157 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.076158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.076158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.076257 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.076558 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:04.076563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:04.076674 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 17:48:04.076792 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 17:48:04.076949 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260808 17:48:04.077880 18891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.62.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.252@KRBTEST.COM" created.
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211284, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
I20260808 17:48:04.085749 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.078153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38322)
0808 17:48:04.078201 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.078211 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:04.078250 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:04.078617 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:04.078617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:04.078756 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:04.078804 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.078805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.079809 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.079810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.080303 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.080304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.080782 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.080783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.080850 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.081262 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:04.081267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:04.082684 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:04.084591 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.084592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.084593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.084826 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.085071 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.085072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.085073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.085152 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.085414 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:04.085415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:04.085454 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:48:04.085529 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 17:48:04.085632 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:48:04.085902 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.078253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.29.62.253:33073)
0808 17:48:04.078466 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.078485 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.078502 (+ 17us) server_negotiation.cc:407] Connection header received
0808 17:48:04.078520 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.078522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.078532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.078617 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.078892 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.079718 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.080396 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.080722 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.080786 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.082771 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.082780 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.082780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.082788 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.084512 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.084902 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.084905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.084905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.084993 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.085218 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.085218 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.085218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.085340 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.085432 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.085570 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.085685 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
I20260808 17:48:04.086513 18891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:48:04.086889 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.086774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41390)
0808 17:48:04.086825 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.086868 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 17:48:04.087023 18815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
I20260808 17:48:04.087471 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.087373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41402)
0808 17:48:04.087421 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.087447 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 17:48:04.087589 18891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:44939: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab.
Entry for principal kudu/127.29.62.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab.
I20260808 17:48:04.097550 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.252:44939
--webserver_interface=127.29.62.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab
--principal=kudu/127.29.62.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:33565,127.29.62.253:33073,127.29.62.252:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:04.124747 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } attempt: 3
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211283, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
I20260808 17:48:04.132418 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.125250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.29.62.253:33073)
0808 17:48:04.125327 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.125328 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.125338 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:48:04.125366 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.125367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.125378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.125439 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.125837 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.126361 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.127433 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.127658 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.127715 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.129763 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.129765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.129765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.129778 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.131251 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.131606 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.131607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.131608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.131666 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.131884 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.131885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.131885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.131996 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.132072 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.132230 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.132314 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:48:04.132418 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.125183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38326)
0808 17:48:04.125252 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.125264 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:04.125310 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:04.125477 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:04.125478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:04.125668 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:04.125753 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.125755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.126493 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.126494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.127335 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.127338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.127734 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.127736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.127820 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.128286 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:04.128291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:04.129665 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:04.131340 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.131341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.131342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.131528 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.131745 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.131746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.131746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.131811 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.132065 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:04.132066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:04.132116 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:48:04.132204 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 17:48:04.132314 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 17:48:04.133013 18781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:48:04.133394 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.133280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41416)
0808 17:48:04.133331 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.133359 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:48:04.133781 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.133696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41428)
0808 17:48:04.133741 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.133765 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 17:48:04.133894 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:44939: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
I20260808 17:48:04.136799 18891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } attempt: 1
I20260808 17:48:04.137131 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.137026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41442)
0808 17:48:04.137076 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.137109 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:48:04.137557 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.137467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41454)
0808 17:48:04.137516 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.137540 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 17:48:04.137671 18891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:44939: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
I20260808 17:48:04.165026 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } attempt: 1
I20260808 17:48:04.165506 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.165362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41466)
0808 17:48:04.165426 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.165464 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 17:48:04.165901 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.165801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41472)
0808 17:48:04.165851 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.165877 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 17:48:04.166018 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:44939: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
W20260808 17:48:04.188316 18901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:04.188575 18901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:04.188599 18901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:04.189718 18901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:04.189756 18901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:04.189770 18901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:04.189783 18901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:04.189795 18901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:04.191380 18901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:33565,127.29.62.253:33073,127.29.62.252:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab
--principal=kudu/127.29.62.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.252:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.62.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:04.191591 18901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:04.191763 18901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:04.196596 18909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:04.196628 18907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:04.196623 18912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:04.197036 18901 server_base.cc:1034] running on GCE node
I20260808 17:48:04.197286 18901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:04.197584 18901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:04.198782 18901 hybrid_clock.cc:648] HybridClock initialized: now 1786211284198769 us; error 24 us; skew 500 ppm
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211284, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:04.201377 18901 init.cc:377] Logged in from keytab as kudu/127.29.62.252@KRBTEST.COM (short username kudu)
I20260808 17:48:04.201844 18901 webserver.cc:460] Webserver started at http://127.29.62.252:36001/ using document root <none> and password file <none>
I20260808 17:48:04.202112 18901 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:04.202188 18901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:04.202338 18901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:04.203500 18901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/data/instance:
uuid: "f35dd11af6b34ee3865ef1646b59c481"
format_stamp: "Formatted at 2026-08-08 17:48:04 on dist-test-slave-tk9z"
server_key: "7051d6ef5ff9ad0d3a33e8f41529305a"
server_key_iv: "8b4fc6d584e24a80b77f659be7a0c833"
server_key_version: "encryptionkey@0"
I20260808 17:48:04.203877 18901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/wal/instance:
uuid: "f35dd11af6b34ee3865ef1646b59c481"
format_stamp: "Formatted at 2026-08-08 17:48:04 on dist-test-slave-tk9z"
server_key: "7051d6ef5ff9ad0d3a33e8f41529305a"
server_key_iv: "8b4fc6d584e24a80b77f659be7a0c833"
server_key_version: "encryptionkey@0"
I20260808 17:48:04.205251 18901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:48:04.205819 18891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } attempt: 2
I20260808 17:48:04.206048 18923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:04.206234 18901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:04.206251 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.206116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41478)
0808 17:48:04.206173 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.206209 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 17:48:04.206357 18901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/wal
uuid: "f35dd11af6b34ee3865ef1646b59c481"
format_stamp: "Formatted at 2026-08-08 17:48:04 on dist-test-slave-tk9z"
server_key: "7051d6ef5ff9ad0d3a33e8f41529305a"
server_key_iv: "8b4fc6d584e24a80b77f659be7a0c833"
server_key_version: "encryptionkey@0"
I20260808 17:48:04.206462 18901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:04.206647 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.206529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41486)
0808 17:48:04.206575 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.206612 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 17:48:04.206745 18891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:44939: Network error: Client connection negotiation failed: client connection to 127.29.62.252:44939: connect: Connection refused (error 111)
I20260808 17:48:04.224529 18901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:04.226640 18901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:04.226799 18901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:04.230804 18901 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.252:44939
I20260808 17:48:04.230860 19004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.252:44939 every 8 connection(s)
I20260808 17:48:04.231237 18901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
I20260808 17:48:04.232215 19005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:04.233568 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 18901
I20260808 17:48:04.233676 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786211125631729-29947-0/minicluster-data/master-2/wal/instance
I20260808 17:48:04.233878 29947 external_mini_cluster.cc:1428] Setting key 5a7bfcc575d387271019c2de3f031a70
I20260808 17:48:04.234115 19005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211284, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
I20260808 17:48:04.241199 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } attempt: 2
I20260808 17:48:04.244995 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.234868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:33565 (local address 127.0.0.1:53840)
0808 17:48:04.235004 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.235086 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:04.235317 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:04.235588 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:04.235591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:04.236015 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:04.236240 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.236246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.236941 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.236942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.237897 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.237900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.238457 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.238458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.238628 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.239148 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:04.239156 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:04.240809 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:04.243369 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.243376 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.243378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.243705 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.243922 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.243923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.243923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.244060 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.244407 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:04.244412 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:04.244520 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 17:48:04.244609 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 17:48:04.244764 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260808 17:48:04.245258 19008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.234456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41490 (local address 127.29.62.252:44939)
0808 17:48:04.234627 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.234636 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.234671 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:48:04.235312 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.235322 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.235348 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.235455 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.236406 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.237365 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.238313 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.238765 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.238864 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.241426 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.241434 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.241434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.241441 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.243595 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.244023 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.244026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.244026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.244083 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.244271 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.244272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.244272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.244524 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.244670 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.245076 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.245156 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260808 17:48:04.245429 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.234949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.29.62.254:33565)
0808 17:48:04.235001 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.235003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.235165 (+ 162us) server_negotiation.cc:407] Connection header received
0808 17:48:04.235438 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.235440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.235453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.235487 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.236357 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.236858 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.237988 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.238321 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.238443 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.240906 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.240907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.240908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.240915 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.243279 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.243785 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.243786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.243787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.243845 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.244145 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.244146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.244146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.244312 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.244380 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.245242 (+ 862us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.245339 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:48:04.245792 19005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211283, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211284, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
I20260808 17:48:04.250705 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.241465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41496)
0808 17:48:04.241538 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.241551 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:04.241594 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:04.241936 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:04.241938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:04.242097 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:04.242141 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.242142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.243158 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.243159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.243924 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.243928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.244386 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.244387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.244447 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.244843 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:04.244849 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:04.246743 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:04.249222 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.249223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.249223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.249724 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.249970 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.249970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.249971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.250040 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.250362 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:04.250363 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:04.250430 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 17:48:04.250485 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:48:04.250579 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 17:48:04.251174 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.241574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.29.62.252:44939)
0808 17:48:04.241734 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.241736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.241744 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:48:04.241766 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.241767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.241780 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.241902 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.242226 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.243062 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.244001 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.244296 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.244384 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.246834 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.246836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.246836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.246843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.249150 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.249813 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.249814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.249814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.249873 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.250122 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.250123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.250124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.250276 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.250919 (+ 643us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.250960 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.251039 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:04.253113 18781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f: Bootstrap starting.
I20260808 17:48:04.253576 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.246030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:33073 (local address 127.0.0.1:38334)
0808 17:48:04.246068 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.246077 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:04.246111 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:04.246595 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:04.246596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:04.246753 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:04.246811 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.246812 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.247695 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.247696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.248177 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.248179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.248560 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.248562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.248632 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.249337 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:04.249342 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:04.250582 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:04.252457 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.252458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.252458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.252635 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.252870 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.252871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.252871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.252939 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.253263 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:04.253264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:04.253310 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 17:48:04.253389 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:48:04.253487 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 17:48:04.253593 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.246405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38334 (local address 127.29.62.253:33073)
0808 17:48:04.246458 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.246460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.246469 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:48:04.246495 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.246496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.246506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.246570 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.246878 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.247601 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.248266 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.248485 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.248563 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.250674 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.250676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.250677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.250685 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.252383 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.252712 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.252713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.252713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.252780 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.253010 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.253011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.253012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.253161 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.253233 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.253431 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.253506 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:48:04.253963 18781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:04.254283 19005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:48:04.254338 18781 log.cc:824] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:04.255041 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.246175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.29.62.254:33565)
0808 17:48:04.246212 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.246214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.246222 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:48:04.246288 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.246290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.246301 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.246358 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.246964 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.247639 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.248956 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.249196 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.249270 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.252040 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.252041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.252041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.252048 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.253835 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.254221 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.254222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.254223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.254289 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.254509 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.254510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.254510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.254643 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.254821 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.254872 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.254952 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:48:04.255272 18781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f: No bootstrap required, opened a new log
I20260808 17:48:04.257342 18781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.257486 18781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:04.257520 18781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3766e756f78c45f08194a232fd94f34f, State: Initialized, Role: FOLLOWER
I20260808 17:48:04.257664 18781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.258004 18781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211284, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
I20260808 17:48:04.258901 19017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.259086 19017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:48:04.259949 19037 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:48:04.260051 19037 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
I20260808 17:48:04.261950 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.254551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41512)
0808 17:48:04.254638 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.254648 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:04.254706 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:04.254849 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:04.254850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:04.254996 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:04.255025 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.255027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.255832 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.255833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.256555 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.256557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.256923 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.256924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.256970 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.257370 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:04.257375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:04.258711 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:04.260837 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.260838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.260839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.261046 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.261277 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.261278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.261278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.261331 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.261590 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:04.261590 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:04.261630 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 17:48:04.261703 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 17:48:04.261826 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 17:48:04.262061 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.254592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.29.62.252:44939)
0808 17:48:04.254628 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.254630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.254684 (+ 54us) server_negotiation.cc:407] Connection header received
0808 17:48:04.254765 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.254765 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.254774 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.254798 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.255106 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.255636 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.256627 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.256851 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.256928 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.258813 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.258814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.258814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.258822 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.260724 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.261127 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.261129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.261129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.261198 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.261389 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.261389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.261390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.261508 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.261568 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.261733 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.261918 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":18}
I20260808 17:48:04.260756 18668 master_runner.cc:186] Error getting master registration for 127.29.62.253:33073: OK, instance_id { permanent_uuid: "9663937117f94649a80b50ba836bc78c" instance_seqno: 1786211284062440 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:48:04.263564 18668 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:48:04.263684 19005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481: Bootstrap starting.
I20260808 17:48:04.264483 19005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:04.264819 19005 log.cc:824] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:04.265172 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.256309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.29.62.253:33073)
0808 17:48:04.256346 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.256348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.256356 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:48:04.256377 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.256378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.256386 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.256441 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.257011 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.257664 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.258671 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.258959 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.259271 (+ 312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.261640 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.261641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.261642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.261649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.263418 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.263834 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.263835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.263835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.263895 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.264133 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.264133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.264134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.264270 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.264960 (+ 690us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.265006 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.265085 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:48:04.265772 19005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481: No bootstrap required, opened a new log
I20260808 17:48:04.266676 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:48:04.267833 19005 raft_consensus.cc:348] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.268008 19005 raft_consensus.cc:374] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:04.268093 19005 raft_consensus.cc:729] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35dd11af6b34ee3865ef1646b59c481, State: Initialized, Role: FOLLOWER
I20260808 17:48:04.268330 19005 consensus_queue.cc:260] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.268750 19005 sys_catalog.cc:564] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 17:48:04.271065 19065 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:48:04.271683 19065 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:48:04.271400 19047 sys_catalog.cc:455] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.273032 19047 sys_catalog.cc:458] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:48:04.273192 18901 master_runner.cc:186] Error getting master registration for 127.29.62.253:33073: OK, instance_id { permanent_uuid: "9663937117f94649a80b50ba836bc78c" instance_seqno: 1786211284062440 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:48:04.275517 18901 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:48:04.280145 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.268256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.29.62.253:33073)
0808 17:48:04.268298 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.268299 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.268838 (+ 539us) server_negotiation.cc:407] Connection header received
0808 17:48:04.268940 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.268941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.268953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.268992 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.270024 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.270813 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.271925 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.272211 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.272255 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.276268 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.276270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.276270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.276279 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.278607 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.279266 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.279268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.279268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.279330 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.279542 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.279543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.279544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.279678 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.279752 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.279963 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.280046 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:48:04.280856 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.268370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41514 (local address 127.29.62.252:44939)
0808 17:48:04.268405 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.268407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.268841 (+ 434us) server_negotiation.cc:407] Connection header received
0808 17:48:04.268961 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.268962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.268972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.269008 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.274575 (+ 5567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.275261 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.276237 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.276537 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.276608 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.277981 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.277982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.277982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.277990 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.279633 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.280018 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.280020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.280020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.280076 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.280272 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.280273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.280273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.280413 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.280479 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.280687 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.280766 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:48:04.285336 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.268615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.29.62.254:33565)
0808 17:48:04.268660 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.268662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.269687 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 17:48:04.269795 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.269796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.269806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.269838 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.270712 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.271408 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.272631 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.272916 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.272957 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.282108 (+ 9151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.282110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.282110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.282118 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.283852 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.284385 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.284386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.284386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.284448 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.284688 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.284689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.284689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.284827 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.284927 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.285142 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.285230 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:48:04.297911 18891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } attempt: 3
Aug 08 17:48:04 dist-test-slave-tk9z krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211284, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
I20260808 17:48:04.306227 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.298206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:44939 (local address 127.0.0.1:41518)
0808 17:48:04.298257 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:04.298268 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:04.298305 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:04.298462 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:04.298463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:04.298658 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:04.298712 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.298714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.299677 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.299678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.300492 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:04.300494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.301016 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.301018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:04.301092 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.301627 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:04.301632 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:04.303211 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:04.305111 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.305112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.305113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.305340 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.305561 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:04.305562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:04.305562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:04.305630 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:04.305902 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:04.305903 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:04.305953 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:48:04.306024 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:48:04.306122 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:48:04.306242 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:04.298267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.29.62.252:44939)
0808 17:48:04.298316 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:04.298318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:04.298328 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:48:04.298353 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:04.298354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:04.298363 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:04.298459 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:04.298796 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.299585 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.300585 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:04.300886 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:04.300956 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:04.303318 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:04.303319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:04.303320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:04.303327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:04.305023 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.305422 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.305423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.305424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.305486 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:04.305699 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:04.305700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:04.305700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:04.305836 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:04.305898 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:04.306063 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 17:48:04.306144 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:48:04.307591 19017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:04.307720 19017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.307986 19017 leader_election.cc:288] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9663937117f94649a80b50ba836bc78c (127.29.62.253:33073), f35dd11af6b34ee3865ef1646b59c481 (127.29.62.252:44939)
I20260808 17:48:04.308297 18964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3766e756f78c45f08194a232fd94f34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35dd11af6b34ee3865ef1646b59c481" is_pre_election: true
I20260808 17:48:04.308395 18891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c: Bootstrap starting.
I20260808 17:48:04.308444 18964 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3766e756f78c45f08194a232fd94f34f in term 0.
I20260808 17:48:04.308614 18696 leader_election.cc:302] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3766e756f78c45f08194a232fd94f34f, f35dd11af6b34ee3865ef1646b59c481; no voters:
I20260808 17:48:04.308744 19017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:48:04.308780 19017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:48:04.308908 19017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:04.309738 18891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:04.309965 19017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.310045 18891 log.cc:824] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:04.310083 19017 leader_election.cc:288] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [CANDIDATE]: Term 1 election: Requested vote from peers 9663937117f94649a80b50ba836bc78c (127.29.62.253:33073), f35dd11af6b34ee3865ef1646b59c481 (127.29.62.252:44939)
I20260808 17:48:04.310341 18964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3766e756f78c45f08194a232fd94f34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35dd11af6b34ee3865ef1646b59c481"
I20260808 17:48:04.310441 18964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:04.311249 18964 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3766e756f78c45f08194a232fd94f34f in term 1.
I20260808 17:48:04.311395 18891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c: No bootstrap required, opened a new log
I20260808 17:48:04.311420 18696 leader_election.cc:302] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3766e756f78c45f08194a232fd94f34f, f35dd11af6b34ee3865ef1646b59c481; no voters:
I20260808 17:48:04.312022 19017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:04.312186 19017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [term 1 LEADER]: Becoming Leader. State: Replica: 3766e756f78c45f08194a232fd94f34f, State: Running, Role: LEADER
I20260808 17:48:04.312266 19017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.312646 19017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3766e756f78c45f08194a232fd94f34f. Latest consensus state: current_term: 1 leader_uuid: "3766e756f78c45f08194a232fd94f34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.312709 19017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:04.312933 19071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:48:04.313133 19071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:48:04.313369 18891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.313515 18891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:04.313549 18891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9663937117f94649a80b50ba836bc78c, State: Initialized, Role: FOLLOWER
I20260808 17:48:04.313654 18891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } }
I20260808 17:48:04.313931 18891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:48:04.314421 18855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3766e756f78c45f08194a232fd94f34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9663937117f94649a80b50ba836bc78c"
I20260808 17:48:04.314531 18855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:04.315042 18964 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [term 1 FOLLOWER]: Refusing update from remote peer 3766e756f78c45f08194a232fd94f34f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:04.315331 18855 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3766e756f78c45f08194a232fd94f34f in term 1.
I20260808 17:48:04.315330 19017 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:04.315452 18856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3766e756f78c45f08194a232fd94f34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9663937117f94649a80b50ba836bc78c" is_pre_election: true
I20260808 17:48:04.315526 18856 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3766e756f78c45f08194a232fd94f34f in term 1. Re-sending same reply.
I20260808 17:48:04.316537 19047 sys_catalog.cc:455] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3766e756f78c45f08194a232fd94f34f. Latest consensus state: current_term: 1 leader_uuid: "3766e756f78c45f08194a232fd94f34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.316623 19047 sys_catalog.cc:458] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:48:04.317018 18856 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [term 1 FOLLOWER]: Refusing update from remote peer 3766e756f78c45f08194a232fd94f34f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:04.317260 19017 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:04.319063 19017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3766e756f78c45f08194a232fd94f34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.319137 19017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:04.320502 19017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3766e756f78c45f08194a232fd94f34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.320608 19017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:04.320920 19047 sys_catalog.cc:455] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3766e756f78c45f08194a232fd94f34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.321005 19047 sys_catalog.cc:458] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:48:04.321550 19105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3766e756f78c45f08194a232fd94f34f. Latest consensus state: current_term: 1 leader_uuid: "3766e756f78c45f08194a232fd94f34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.321707 19105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:48:04.321803 19105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3766e756f78c45f08194a232fd94f34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.322170 19105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [sys.catalog]: This master's current role is: FOLLOWER
W20260808 17:48:04.322512 19097 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:48:04.322567 19097 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:48:04.322991 19074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3766e756f78c45f08194a232fd94f34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3766e756f78c45f08194a232fd94f34f" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 33565 } } peers { permanent_uuid: "9663937117f94649a80b50ba836bc78c" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 33073 } } peers { permanent_uuid: "f35dd11af6b34ee3865ef1646b59c481" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 44939 } } }
I20260808 17:48:04.323073 19074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:48:04.323146 19075 mvcc.cc:205] Tried to move back new op lower bound from 7316321420552110080 to 7316321420543643648. Current Snapshot: MvccSnapshot[applied={T|T < 7316321420552110080}]
I20260808 17:48:04.323714 19079 mvcc.cc:205] Tried to move back new op lower bound from 7316321420552110080 to 7316321420543643648. Current Snapshot: MvccSnapshot[applied={T|T < 7316321420552110080}]
I20260808 17:48:04.327032 19102 mvcc.cc:205] Tried to move back new op lower bound from 7316321420552110080 to 7316321420543643648. Current Snapshot: MvccSnapshot[applied={T|T < 7316321420552110080}]
I20260808 17:48:04.327250 19071 catalog_manager.cc:1347] Generated new cluster ID: 76e0d4a9009c4939ad736cf20d0e70f1
I20260808 17:48:04.327306 19071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:48:04.338004 19071 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:48:04.338909 19071 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:48:04.346122 19071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3766e756f78c45f08194a232fd94f34f: Generated new TSK 0
I20260808 17:48:04.346396 19071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:48:05.272392 19065 catalog_manager.cc:1259] Loaded cluster ID: 76e0d4a9009c4939ad736cf20d0e70f1
I20260808 17:48:05.272550 19065 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481: loading cluster ID for follower catalog manager: success
I20260808 17:48:05.273685 19065 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481: acquiring CA information for follower catalog manager: success
I20260808 17:48:05.274206 19065 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f35dd11af6b34ee3865ef1646b59c481: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:48:05.323220 19097 catalog_manager.cc:1259] Loaded cluster ID: 76e0d4a9009c4939ad736cf20d0e70f1
I20260808 17:48:05.323349 19097 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c: loading cluster ID for follower catalog manager: success
I20260808 17:48:05.324519 19097 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c: acquiring CA information for follower catalog manager: success
I20260808 17:48:05.324982 19097 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9663937117f94649a80b50ba836bc78c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:48:05.407518 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 18668
I20260808 17:48:05.430353 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 18792
I20260808 17:48:05.451417 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 18901
2026-08-08T17:48:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3089 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:48:05 dist-test-slave-tk9z krb5kdc[19131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:48:05 dist-test-slave-tk9z krb5kdc[19131](info): set up 2 sockets
Aug 08 17:48:05 dist-test-slave-tk9z krb5kdc[19131](info): commencing operation
krb5kdc: starting...
W20260808 17:48:06.819195 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:48:06 dist-test-slave-tk9z krb5kdc[19131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:48:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:48:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:48:06.950217 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:39463
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:39463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:07.035351 19154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:07.035571 19154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:07.035594 19154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:07.036796 19154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:07.036842 19154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:07.036856 19154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:07.036870 19154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:07.036921 19154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:07.038466 19154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:39463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:39463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:07.038727 19154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:07.038970 19154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:07.043978 19165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:07.044181 19154 server_base.cc:1034] running on GCE node
W20260808 17:48:07.043978 19168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:07.043964 19166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:07.044561 19154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:07.044872 19154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:07.046020 19154 hybrid_clock.cc:648] HybridClock initialized: now 1786211287045995 us; error 51 us; skew 500 ppm
Aug 08 17:48:07 dist-test-slave-tk9z krb5kdc[19131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211287, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:07.048651 19154 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:48:07.049105 19154 webserver.cc:460] Webserver started at http://127.29.62.254:33389/ using document root <none> and password file <none>
I20260808 17:48:07.049340 19154 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:07.049408 19154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:07.049569 19154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:07.050769 19154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "3ee49b34bb424d07b1745c0802a392da"
format_stamp: "Formatted at 2026-08-08 17:48:07 on dist-test-slave-tk9z"
server_key: "b1facbfa065c88e01d7905e0abf068f4"
server_key_iv: "61b48027b8fc1a8dbd21356ef2663806"
server_key_version: "encryptionkey@0"
I20260808 17:48:07.051155 19154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "3ee49b34bb424d07b1745c0802a392da"
format_stamp: "Formatted at 2026-08-08 17:48:07 on dist-test-slave-tk9z"
server_key: "b1facbfa065c88e01d7905e0abf068f4"
server_key_iv: "61b48027b8fc1a8dbd21356ef2663806"
server_key_version: "encryptionkey@0"
I20260808 17:48:07.052546 19154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:48:07.053390 19178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:07.053575 19154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:07.053692 19154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "3ee49b34bb424d07b1745c0802a392da"
format_stamp: "Formatted at 2026-08-08 17:48:07 on dist-test-slave-tk9z"
server_key: "b1facbfa065c88e01d7905e0abf068f4"
server_key_iv: "61b48027b8fc1a8dbd21356ef2663806"
server_key_version: "encryptionkey@0"
I20260808 17:48:07.053773 19154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:07.067795 19154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:07.070142 19154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:07.070322 19154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:07.074527 19154 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:39463
I20260808 17:48:07.074584 19258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:39463 every 8 connection(s)
I20260808 17:48:07.074968 19154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:48:07.075896 19259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:07.076339 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 19154
I20260808 17:48:07.076467 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:48:07.076681 29947 external_mini_cluster.cc:1428] Setting key 9bd0e1d02c76a2ca37532fca81da42de
I20260808 17:48:07.079102 19259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da: Bootstrap starting.
I20260808 17:48:07.079720 19259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:07.080056 19259 log.cc:824] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:07.080848 19259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da: No bootstrap required, opened a new log
Aug 08 17:48:07 dist-test-slave-tk9z krb5kdc[19131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:07.083498 19259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee49b34bb424d07b1745c0802a392da" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39463 } }
I20260808 17:48:07.083705 19259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:07.083782 19259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee49b34bb424d07b1745c0802a392da, State: Initialized, Role: FOLLOWER
I20260808 17:48:07.083932 19259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee49b34bb424d07b1745c0802a392da" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39463 } }
I20260808 17:48:07.084023 19259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:48:07.084069 19259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:48:07.084117 19259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:07.084883 19259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee49b34bb424d07b1745c0802a392da" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39463 } }
I20260808 17:48:07.085008 19259 leader_election.cc:302] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ee49b34bb424d07b1745c0802a392da; no voters:
I20260808 17:48:07.085168 19259 leader_election.cc:288] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:48:07.085248 19266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:07.085419 19266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [term 1 LEADER]: Becoming Leader. State: Replica: 3ee49b34bb424d07b1745c0802a392da, State: Running, Role: LEADER
I20260808 17:48:07.085462 19259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:48:07.085551 19266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee49b34bb424d07b1745c0802a392da" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39463 } }
I20260808 17:48:07.085958 19268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ee49b34bb424d07b1745c0802a392da. Latest consensus state: current_term: 1 leader_uuid: "3ee49b34bb424d07b1745c0802a392da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee49b34bb424d07b1745c0802a392da" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39463 } } }
I20260808 17:48:07.086100 19268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:07.086627 19276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:48:07.086601 19267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ee49b34bb424d07b1745c0802a392da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee49b34bb424d07b1745c0802a392da" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39463 } } }
I20260808 17:48:07.086900 19267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:07.087365 19276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:48:07.088423 19263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:07.077242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44560 (local address 127.29.62.254:39463)
0808 17:48:07.077439 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:07.077448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:07.077489 (+ 41us) server_negotiation.cc:407] Connection header received
0808 17:48:07.077891 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:07.077897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:07.077942 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:07.078059 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:07.078919 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:07.079856 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:07.080819 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:07.081263 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:07.081345 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:07.084187 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:07.084191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:07.084192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:07.084204 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:07.086441 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:07.086902 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:07.086905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:07.086905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:07.087035 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:07.087274 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:07.087275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:07.087275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:07.087654 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:07.087811 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:07.088052 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 17:48:07.088196 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260808 17:48:07.089298 19276 catalog_manager.cc:1347] Generated new cluster ID: 701ad92d4521475ba6d22ff4d74993db
I20260808 17:48:07.089347 19276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:48:07.089439 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:48:07.101971 19276 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:48:07.102833 19276 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:48:07.112017 19276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ee49b34bb424d07b1745c0802a392da: Generated new TSK 0
I20260808 17:48:07.112241 19276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:48:07.143680 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 19154
2026-08-08T17:48:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1679 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:48:07 dist-test-slave-tk9z krb5kdc[19299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:48:07 dist-test-slave-tk9z krb5kdc[19299](info): set up 2 sockets
Aug 08 17:48:07 dist-test-slave-tk9z krb5kdc[19299](info): commencing operation
krb5kdc: starting...
W20260808 17:48:08.377978 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:48:08 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:48:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:48:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:48:08.497853 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40337
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:08.583105 19320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:08.583333 19320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:08.583360 19320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:08.584649 19320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:08.584697 19320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:08.584714 19320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:08.584728 19320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:08.584743 19320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:08.586416 19320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:08.586700 19320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:08.586961 19320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:08.592190 19326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:08.592221 19331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:08.592217 19327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:08.592412 19320 server_base.cc:1034] running on GCE node
I20260808 17:48:08.592720 19320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:08.593011 19320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:08.594151 19320 hybrid_clock.cc:648] HybridClock initialized: now 1786211288594139 us; error 28 us; skew 500 ppm
Aug 08 17:48:08 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211288, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:08.599922 19320 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:48:08.600358 19320 webserver.cc:460] Webserver started at http://127.29.62.254:39391/ using document root <none> and password file <none>
I20260808 17:48:08.600589 19320 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:08.600658 19320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:08.600812 19320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:08.601845 19320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "64541828ad6041f2bc0982bdb783e480"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "7fd07c9bdad99cab756ce6ecc7dcbb89"
server_key_iv: "362c6ba3ce9509bb55329a84110e856c"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.602216 19320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "64541828ad6041f2bc0982bdb783e480"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "7fd07c9bdad99cab756ce6ecc7dcbb89"
server_key_iv: "362c6ba3ce9509bb55329a84110e856c"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.603588 19320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:48:08.604343 19341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:08.604621 19320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:08.604732 19320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "64541828ad6041f2bc0982bdb783e480"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "7fd07c9bdad99cab756ce6ecc7dcbb89"
server_key_iv: "362c6ba3ce9509bb55329a84110e856c"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.604807 19320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:08.623011 19320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:08.625361 19320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:08.625545 19320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:08.630424 19320 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40337
I20260808 17:48:08.630460 19419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40337 every 8 connection(s)
I20260808 17:48:08.630882 19320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:48:08.631979 19420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:08.632920 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 19320
I20260808 17:48:08.633067 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:48:08.633276 29947 external_mini_cluster.cc:1428] Setting key 55fa56b1f0f3b6815f46ccc6edf691a3
I20260808 17:48:08.634771 19420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Bootstrap starting.
I20260808 17:48:08.635545 19420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:08.635869 19420 log.cc:824] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:08.636616 19420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: No bootstrap required, opened a new log
I20260808 17:48:08.639096 19420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:08.639303 19420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:08.639344 19420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64541828ad6041f2bc0982bdb783e480, State: Initialized, Role: FOLLOWER
I20260808 17:48:08.639508 19420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:08.639611 19420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:48:08.639652 19420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:48:08.639701 19420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:08.640348 19420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
Aug 08 17:48:08 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:08.640480 19420 leader_election.cc:302] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64541828ad6041f2bc0982bdb783e480; no voters:
I20260808 17:48:08.640693 19420 leader_election.cc:288] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:48:08.640717 19431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:08.640841 19431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 1 LEADER]: Becoming Leader. State: Replica: 64541828ad6041f2bc0982bdb783e480, State: Running, Role: LEADER
I20260808 17:48:08.641003 19420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:48:08.641022 19431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:08.641497 19432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64541828ad6041f2bc0982bdb783e480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } } }
I20260808 17:48:08.642058 19432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:08.641534 19433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64541828ad6041f2bc0982bdb783e480. Latest consensus state: current_term: 1 leader_uuid: "64541828ad6041f2bc0982bdb783e480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } } }
I20260808 17:48:08.642354 19433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:08.642601 19445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:48:08.643272 19445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:48:08.644989 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:08.633845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.29.62.254:40337)
0808 17:48:08.634031 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:08.634039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:08.634064 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:48:08.634562 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:08.634569 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:08.634620 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:08.634737 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:08.635563 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:08.636555 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:08.637646 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:08.638065 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:08.638150 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:08.641039 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:08.641044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:08.641044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:08.641058 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:08.643345 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:08.643750 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:08.643754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:08.643755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:08.643855 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:08.644075 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:08.644076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:08.644076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:08.644394 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:08.644554 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:08.644666 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 17:48:08.644813 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260808 17:48:08.646801 19445 catalog_manager.cc:1347] Generated new cluster ID: 71f93dc438744bdea73914f558733be1
I20260808 17:48:08.646860 19445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:48:08.654287 19445 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:48:08.655046 19445 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:48:08.665433 19445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Generated new TSK 0
I20260808 17:48:08.665637 19445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:48:08.697141 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40337
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:08.780951 19462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:08.781159 19462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:08.781183 19462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:08.782322 19462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:08.782411 19462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:08.782459 19462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:48:08.783874 19462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:08.784071 19462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:08.784255 19462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:08.789302 19470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:08.789289 19474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:08.789296 19471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:08.789626 19462 server_base.cc:1034] running on GCE node
I20260808 17:48:08.789829 19462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:08.790098 19462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:08.791251 19462 hybrid_clock.cc:648] HybridClock initialized: now 1786211288791230 us; error 34 us; skew 500 ppm
Aug 08 17:48:08 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211288, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:08.793708 19462 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:48:08.794139 19462 webserver.cc:460] Webserver started at http://127.29.62.193:44337/ using document root <none> and password file <none>
I20260808 17:48:08.794418 19462 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:08.794493 19462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:08.794628 19462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:08.795656 19462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "92a1f708b8dc32b452e55a61aca89a86"
server_key_iv: "6d01597abac138b0cd5f588a9387c0e0"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.796034 19462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "92a1f708b8dc32b452e55a61aca89a86"
server_key_iv: "6d01597abac138b0cd5f588a9387c0e0"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.797317 19462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:48:08.798004 19485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:08.798166 19462 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:48:08.798270 19462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "92a1f708b8dc32b452e55a61aca89a86"
server_key_iv: "6d01597abac138b0cd5f588a9387c0e0"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.798343 19462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:08.835129 19462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:08.837013 19462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:08.837181 19462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:08.837430 19462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:08.837755 19462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:08.837811 19462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:08.837852 19462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:08.837882 19462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:08.844134 19462 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:44639
I20260808 17:48:08.844218 19670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:44639 every 8 connection(s)
I20260808 17:48:08.844552 19462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
Aug 08 17:48:08 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211288, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:08.853329 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 19462
I20260808 17:48:08.853462 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:48:08.853703 29947 external_mini_cluster.cc:1428] Setting key b88bdd2292f6189e78cf704b8682b0ac
I20260808 17:48:08.854633 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:08.845299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:36689 (local address 127.29.62.254:40337)
0808 17:48:08.845351 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:08.845354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:08.845548 (+ 194us) server_negotiation.cc:407] Connection header received
0808 17:48:08.846117 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:08.846118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:08.846131 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:08.846197 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:08.847265 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:08.847795 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:08.848810 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:08.849033 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:08.849114 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:08.851710 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:08.851712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:08.851712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:08.851720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:08.852931 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:08.853481 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:08.853483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:08.853483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:08.853560 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:08.853844 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:08.853845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:08.853846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:08.854016 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:08.854093 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:08.854436 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 17:48:08.854531 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:48:08.854822 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:08.845255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:36689)
0808 17:48:08.845400 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:08.845452 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:08.846002 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:08.846274 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:08.846279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:08.846950 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:08.847158 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:08.847163 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:08.847897 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:08.847899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:08.848712 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:08.848714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:08.849125 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:08.849127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:08.849267 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:08.849868 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:08.849876 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:08.851610 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:08.853045 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:08.853053 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:08.853055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:08.853375 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:08.853644 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:08.853645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:08.853646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:08.853758 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:08.854107 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:08.854115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:08.854268 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 17:48:08.854411 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 17:48:08.854594 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260808 17:48:08.855435 19671 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40337
I20260808 17:48:08.855544 19671 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:08.855744 19671 heartbeater.cc:511] Master 127.29.62.254:40337 requested a full tablet report, sending...
I20260808 17:48:08.856246 19363 ts_manager.cc:194] Registered new tserver with Master: 6fbba6faf9e24bacb7f6f0b8b19cd2e1 (127.29.62.193:44639)
I20260808 17:48:08.856916 19363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:36689
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:48:08.880797 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40337
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:08.966993 19679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:08.967201 19679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:08.967227 19679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:08.968397 19679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:08.968449 19679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:08.968477 19679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:48:08.969957 19679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:08.970220 19679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:08.970537 19679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:08.975631 19688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:08.975663 19684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:08.975698 19679 server_base.cc:1034] running on GCE node
W20260808 17:48:08.975658 19685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:08.976130 19679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:08.976424 19679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:08.977591 19679 hybrid_clock.cc:648] HybridClock initialized: now 1786211288977578 us; error 43 us; skew 500 ppm
Aug 08 17:48:08 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211288, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:08.980283 19679 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:48:08.980791 19679 webserver.cc:460] Webserver started at http://127.29.62.194:34831/ using document root <none> and password file <none>
I20260808 17:48:08.981081 19679 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:08.981159 19679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:08.981325 19679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:08.982591 19679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "8e03a3df12d643e49ae99ed29c97043e"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "484532d4d07984cd48011d8f0b7e220f"
server_key_iv: "fa92e4629b06b105523ea3cac61881f4"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.983011 19679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "8e03a3df12d643e49ae99ed29c97043e"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "484532d4d07984cd48011d8f0b7e220f"
server_key_iv: "fa92e4629b06b105523ea3cac61881f4"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.984506 19679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:48:08.985291 19696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:08.985486 19679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:48:08.985606 19679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "8e03a3df12d643e49ae99ed29c97043e"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "484532d4d07984cd48011d8f0b7e220f"
server_key_iv: "fa92e4629b06b105523ea3cac61881f4"
server_key_version: "encryptionkey@0"
I20260808 17:48:08.985711 19679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:08.999902 19679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:09.001847 19679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:09.001968 19679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:09.002270 19679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:09.002650 19679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:09.002708 19679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.002749 19679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:09.002779 19679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.008922 19679 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:45129
I20260808 17:48:09.008983 19876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:45129 every 8 connection(s)
I20260808 17:48:09.009358 19679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211288, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:09.016515 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 19679
I20260808 17:48:09.016642 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:48:09.016853 29947 external_mini_cluster.cc:1428] Setting key 626f18fefa53aee7622b37a521540825
I20260808 17:48:09.019199 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.010118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:57077 (local address 127.29.62.254:40337)
0808 17:48:09.010193 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.010210 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.010445 (+ 235us) server_negotiation.cc:407] Connection header received
0808 17:48:09.010994 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.010996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.011008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.011062 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:09.012034 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.012538 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.013544 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.013750 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.013838 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.016002 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:09.016004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:09.016004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:09.016012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:09.017775 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.018175 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.018177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.018177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.018237 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.018546 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.018547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.018548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.018669 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:09.018747 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.019026 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.019107 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:48:09.019399 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.010108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:57077)
0808 17:48:09.010292 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.010328 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.010885 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.011135 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.011139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.011726 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:09.011927 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.011932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.012636 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.012637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.013445 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.013447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.013849 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.013851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.013976 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.014545 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:09.014553 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:09.015902 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:09.017863 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.017870 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.017872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.018097 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.018311 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.018312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.018313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.018476 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.018737 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:09.018744 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:09.018886 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.018991 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.019146 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260808 17:48:09.019986 19879 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40337
I20260808 17:48:09.020095 19879 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:09.020270 19879 heartbeater.cc:511] Master 127.29.62.254:40337 requested a full tablet report, sending...
I20260808 17:48:09.020737 19363 ts_manager.cc:194] Registered new tserver with Master: 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129)
I20260808 17:48:09.021095 19363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:57077
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:48:09.043200 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40337
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:09.126078 19888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:09.126313 19888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:09.126353 19888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:09.127496 19888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:09.127564 19888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:09.127614 19888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:48:09.129083 19888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:09.129359 19888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:09.129595 19888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:09.135289 19902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.135311 19907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.135288 19900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:09.135547 19888 server_base.cc:1034] running on GCE node
I20260808 17:48:09.135788 19888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:09.136058 19888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:09.137233 19888 hybrid_clock.cc:648] HybridClock initialized: now 1786211289137224 us; error 37 us; skew 500 ppm
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:09.139889 19888 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:48:09.140304 19888 webserver.cc:460] Webserver started at http://127.29.62.195:45685/ using document root <none> and password file <none>
I20260808 17:48:09.140532 19888 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:09.140574 19888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:09.140668 19888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:09.141891 19888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "30cfdc793f7d409ca81ce400d68c168d"
format_stamp: "Formatted at 2026-08-08 17:48:09 on dist-test-slave-tk9z"
server_key: "2db4c8412e9cffa7a1a217998bd4e816"
server_key_iv: "a81dbe754fbbb2aebfc9af904e382817"
server_key_version: "encryptionkey@0"
I20260808 17:48:09.142251 19888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "30cfdc793f7d409ca81ce400d68c168d"
format_stamp: "Formatted at 2026-08-08 17:48:09 on dist-test-slave-tk9z"
server_key: "2db4c8412e9cffa7a1a217998bd4e816"
server_key_iv: "a81dbe754fbbb2aebfc9af904e382817"
server_key_version: "encryptionkey@0"
I20260808 17:48:09.143747 19888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:48:09.144507 19915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.144707 19888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:48:09.144809 19888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "30cfdc793f7d409ca81ce400d68c168d"
format_stamp: "Formatted at 2026-08-08 17:48:09 on dist-test-slave-tk9z"
server_key: "2db4c8412e9cffa7a1a217998bd4e816"
server_key_iv: "a81dbe754fbbb2aebfc9af904e382817"
server_key_version: "encryptionkey@0"
I20260808 17:48:09.144901 19888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:09.178318 19888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:09.180327 19888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:09.180485 19888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:09.180758 19888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:09.181087 19888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:09.181123 19888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.181145 19888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:09.181180 19888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.189081 19888 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:45017
I20260808 17:48:09.189146 20091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:45017 every 8 connection(s)
I20260808 17:48:09.189572 19888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:09.199051 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 19888
I20260808 17:48:09.199165 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:48:09.199378 29947 external_mini_cluster.cc:1428] Setting key 079ee26b04b6d58d8b883db3a1fec23c
I20260808 17:48:09.200239 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.190765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:51807 (local address 127.29.62.254:40337)
0808 17:48:09.190812 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.190814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.190825 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:48:09.191444 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.191446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.191460 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.191552 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:09.192749 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.193360 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.194890 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.195167 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.195226 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.197421 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:09.197422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:09.197423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:09.197429 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:09.198716 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.199118 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.199119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.199120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.199191 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.199471 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.199472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.199478 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.199618 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:09.199712 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.200072 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.200156 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:48:09.200492 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.190413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:51807)
0808 17:48:09.190616 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.190683 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.191277 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.191627 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.191631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.192381 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:09.192638 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.192643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.193464 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.193465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.194792 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.194796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.195245 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.195245 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.195365 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.195927 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:09.195936 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:09.197339 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:09.198793 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.198801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.198802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.199045 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.199274 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.199275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.199275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.199381 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.199703 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:09.199724 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:09.199918 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.200033 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.200198 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
I20260808 17:48:09.201046 20092 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40337
I20260808 17:48:09.201159 20092 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:09.201344 20092 heartbeater.cc:511] Master 127.29.62.254:40337 requested a full tablet report, sending...
I20260808 17:48:09.201750 19363 ts_manager.cc:194] Registered new tserver with Master: 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
I20260808 17:48:09.202078 19363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:51807
I20260808 17:48:09.211347 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:48:09.218482 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.212715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.29.62.254:40337)
0808 17:48:09.212790 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.212793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.212837 (+ 44us) server_negotiation.cc:407] Connection header received
0808 17:48:09.212894 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.212895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.212909 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.212956 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:09.213504 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.213982 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.214710 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.214885 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.214945 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.216068 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:09.216069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:09.216069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:09.216075 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:09.217470 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.217766 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.217767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.217767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.217819 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.217985 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.217985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.217985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.218125 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:09.218202 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.218283 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.218340 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 17:48:09.219910 19363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:48:09.221472 19363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:48:09.228561 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.224066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45017 (local address 127.0.0.1:54336)
0808 17:48:09.224544 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.224577 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.224685 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.224967 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.224970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.224974 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:09.225048 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.225050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.226399 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.226400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.227542 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.227544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.228223 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.228224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.228305 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.228318 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.228427 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.228483 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":20,"thread_start_us":64,"threads_started":1}
I20260808 17:48:09.228572 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.224317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.29.62.195:45017)
0808 17:48:09.224528 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.224533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.224659 (+ 126us) server_negotiation.cc:407] Connection header received
0808 17:48:09.224851 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.224855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.224872 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.224948 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:09.225109 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.226269 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.227624 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.228122 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.228220 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.228282 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.228446 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.228485 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":93,"threads_started":1}
I20260808 17:48:09.228684 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.224181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58724 (local address 127.29.62.194:45129)
0808 17:48:09.224324 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.224329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.224876 (+ 547us) server_negotiation.cc:407] Connection header received
0808 17:48:09.224932 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.224935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.224952 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.225033 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:09.225161 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.225970 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.227467 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.228194 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.228265 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.228319 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.228543 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.228586 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260808 17:48:09.228804 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.224070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:44639 (local address 127.0.0.1:45816)
0808 17:48:09.224931 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.224940 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.224981 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.225322 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.225323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.225325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:09.225363 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.225365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.226579 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.226580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.227582 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.227585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.228519 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.228520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.228573 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.228577 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.228622 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.228724 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260808 17:48:09.229151 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.224068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45129 (local address 127.0.0.1:58724)
0808 17:48:09.224792 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.224802 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.224852 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.225037 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.225038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.225039 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:09.225095 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.225097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.226070 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.226071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.227382 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.227385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.228356 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.228358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.228429 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.228433 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.229045 (+ 612us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.229090 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":21,"thread_start_us":110,"threads_started":1}
I20260808 17:48:09.229450 19993 tablet_service.cc:1467] Processing CreateTablet for tablet dd086e8ee13d471d8b9da70add941b19 (DEFAULT_TABLE table=test-table [id=294bbdeb8c6f44bc91e4a48e1f11ab0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:09.229858 19993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd086e8ee13d471d8b9da70add941b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:09.229918 19773 tablet_service.cc:1467] Processing CreateTablet for tablet dd086e8ee13d471d8b9da70add941b19 (DEFAULT_TABLE table=test-table [id=294bbdeb8c6f44bc91e4a48e1f11ab0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:09.230180 19773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd086e8ee13d471d8b9da70add941b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:09.232484 20117 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: Bootstrap starting.
I20260808 17:48:09.232777 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.224334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.29.62.193:44639)
0808 17:48:09.224529 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.224533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.225168 (+ 635us) server_negotiation.cc:407] Connection header received
0808 17:48:09.225214 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.225216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.225232 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.225307 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:09.225426 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.226481 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.227678 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.228378 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.228437 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.228487 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.232628 (+ 4141us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.232679 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260808 17:48:09.233170 20118 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Bootstrap starting.
I20260808 17:48:09.233214 20117 tablet_bootstrap.cc:654] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:09.233418 19567 tablet_service.cc:1467] Processing CreateTablet for tablet dd086e8ee13d471d8b9da70add941b19 (DEFAULT_TABLE table=test-table [id=294bbdeb8c6f44bc91e4a48e1f11ab0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:09.233731 19567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd086e8ee13d471d8b9da70add941b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:09.233876 20118 tablet_bootstrap.cc:654] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:09.233855 20117 log.cc:824] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:09.234454 20118 log.cc:824] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:09.234730 20117 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: No bootstrap required, opened a new log
I20260808 17:48:09.234827 20117 ts_tablet_manager.cc:1397] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:48:09.235249 20118 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: No bootstrap required, opened a new log
I20260808 17:48:09.235379 20118 ts_tablet_manager.cc:1397] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:48:09.236721 20127 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Bootstrap starting.
I20260808 17:48:09.236860 20117 raft_consensus.cc:348] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.237041 20117 raft_consensus.cc:374] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:09.237092 20117 raft_consensus.cc:729] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30cfdc793f7d409ca81ce400d68c168d, State: Initialized, Role: FOLLOWER
I20260808 17:48:09.237205 20117 consensus_queue.cc:260] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.237445 20127 tablet_bootstrap.cc:654] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:09.237484 20117 ts_tablet_manager.cc:1428] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:48:09.237509 20092 heartbeater.cc:503] Master 127.29.62.254:40337 was elected leader, sending a full tablet report...
I20260808 17:48:09.237818 20118 raft_consensus.cc:348] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.238014 20118 raft_consensus.cc:374] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:09.238065 20118 raft_consensus.cc:729] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e03a3df12d643e49ae99ed29c97043e, State: Initialized, Role: FOLLOWER
I20260808 17:48:09.238063 20127 log.cc:824] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:09.238188 20118 consensus_queue.cc:260] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.238516 20118 ts_tablet_manager.cc:1428] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:48:09.238644 19879 heartbeater.cc:503] Master 127.29.62.254:40337 was elected leader, sending a full tablet report...
I20260808 17:48:09.239172 20127 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: No bootstrap required, opened a new log
I20260808 17:48:09.239240 20127 ts_tablet_manager.cc:1397] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:48:09.240661 20127 raft_consensus.cc:348] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.240780 20127 raft_consensus.cc:374] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:09.240801 20127 raft_consensus.cc:729] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fbba6faf9e24bacb7f6f0b8b19cd2e1, State: Initialized, Role: FOLLOWER
I20260808 17:48:09.240911 20127 consensus_queue.cc:260] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.241225 19671 heartbeater.cc:503] Master 127.29.62.254:40337 was elected leader, sending a full tablet report...
I20260808 17:48:09.241421 20127 ts_tablet_manager.cc:1428] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260808 17:48:09.259991 19881 tablet.cc:2369] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:48:09.325496 20131 raft_consensus.cc:482] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:09.325644 20131 raft_consensus.cc:504] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.326155 20131 leader_election.cc:288] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017), 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129)
I20260808 17:48:09.330403 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.326441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:59339 (local address 127.29.62.195:45017)
0808 17:48:09.326495 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.326498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.326509 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:48:09.326550 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.326551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.326562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.326653 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:09.326870 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.327732 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.329288 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.329943 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.330042 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.330051 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.330236 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.330291 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:48:09.330395 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.326482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:48501 (local address 127.29.62.194:45129)
0808 17:48:09.326529 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.326531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.326671 (+ 140us) server_negotiation.cc:407] Connection header received
0808 17:48:09.326710 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.326711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.326729 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.326779 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:09.326954 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.327743 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.329288 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.329943 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.330036 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.330047 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.330236 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.330289 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:48:09.330402 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.326331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45017 (local address 127.29.62.193:59339)
0808 17:48:09.326423 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.326436 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.326482 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.326675 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.326677 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.326679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:09.326781 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.326783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.327843 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.327843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.329173 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.329179 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.330055 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.330056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.330150 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.330152 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.330210 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.330278 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":83}
I20260808 17:48:09.330417 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.326424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45129 (local address 127.29.62.193:48501)
0808 17:48:09.326583 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.326591 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.326645 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.326804 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.326805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.326807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:09.326877 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.326878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.327843 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.327843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.329173 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.329179 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.330055 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.330056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.330146 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.330150 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.330210 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.330278 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":82,"thread_start_us":105,"threads_started":1}
I20260808 17:48:09.330875 20020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd086e8ee13d471d8b9da70add941b19" candidate_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30cfdc793f7d409ca81ce400d68c168d" is_pre_election: true
I20260808 17:48:09.330881 19801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd086e8ee13d471d8b9da70add941b19" candidate_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e03a3df12d643e49ae99ed29c97043e" is_pre_election: true
I20260808 17:48:09.331069 20020 raft_consensus.cc:2454] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fbba6faf9e24bacb7f6f0b8b19cd2e1 in term 0.
I20260808 17:48:09.331070 19801 raft_consensus.cc:2454] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fbba6faf9e24bacb7f6f0b8b19cd2e1 in term 0.
I20260808 17:48:09.331331 19486 leader_election.cc:302] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30cfdc793f7d409ca81ce400d68c168d, 6fbba6faf9e24bacb7f6f0b8b19cd2e1; no voters:
I20260808 17:48:09.331517 20131 raft_consensus.cc:2781] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:48:09.331560 20131 raft_consensus.cc:482] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:48:09.331575 20131 raft_consensus.cc:3037] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:09.332285 20131 raft_consensus.cc:504] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.332453 20131 leader_election.cc:288] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 election: Requested vote from peers 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017), 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129)
I20260808 17:48:09.332610 20020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd086e8ee13d471d8b9da70add941b19" candidate_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30cfdc793f7d409ca81ce400d68c168d"
I20260808 17:48:09.332626 19801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd086e8ee13d471d8b9da70add941b19" candidate_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e03a3df12d643e49ae99ed29c97043e"
I20260808 17:48:09.332708 20020 raft_consensus.cc:3037] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:09.332715 19801 raft_consensus.cc:3037] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:09.333420 20020 raft_consensus.cc:2454] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fbba6faf9e24bacb7f6f0b8b19cd2e1 in term 1.
I20260808 17:48:09.333593 19801 raft_consensus.cc:2454] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fbba6faf9e24bacb7f6f0b8b19cd2e1 in term 1.
I20260808 17:48:09.333633 19486 leader_election.cc:302] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30cfdc793f7d409ca81ce400d68c168d, 6fbba6faf9e24bacb7f6f0b8b19cd2e1; no voters:
I20260808 17:48:09.333740 20131 raft_consensus.cc:2781] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:09.333950 20131 raft_consensus.cc:686] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 LEADER]: Becoming Leader. State: Replica: 6fbba6faf9e24bacb7f6f0b8b19cd2e1, State: Running, Role: LEADER
I20260808 17:48:09.334055 20131 consensus_queue.cc:237] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.335031 19363 catalog_manager.cc:5495] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fbba6faf9e24bacb7f6f0b8b19cd2e1 (127.29.62.193). New cstate: current_term: 1 leader_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } health_report { overall_health: HEALTHY } } }
W20260808 17:48:09.345278 19672 tablet.cc:2369] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:48:09.345288 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.342878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.29.62.193:44639)
0808 17:48:09.342966 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.342969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.342989 (+ 20us) server_negotiation.cc:407] Connection header received
0808 17:48:09.343039 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.343040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.343059 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.343132 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:48:09.343239 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.343753 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.344460 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.344662 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.344724 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.344863 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:48:09.344969 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:48:09.345045 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.345133 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.345177 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:48:09.347335 20020 raft_consensus.cc:1261] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 1 FOLLOWER]: Refusing update from remote peer 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:09.347359 19801 raft_consensus.cc:1261] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 1 FOLLOWER]: Refusing update from remote peer 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:09.347771 20135 consensus_queue.cc:1035] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:09.347944 20131 consensus_queue.cc:1035] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:09.350446 20139 mvcc.cc:205] Tried to move back new op lower bound from 7316321441163718656 to 7316321441113104384. Current Snapshot: MvccSnapshot[applied={T|T < 7316321441163718656}]
I20260808 17:48:09.350817 20143 mvcc.cc:205] Tried to move back new op lower bound from 7316321441163718656 to 7316321441113104384. Current Snapshot: MvccSnapshot[applied={T|T < 7316321441163718656}]
I20260808 17:48:09.353929 19362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:09.354058 19362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:09.354913 19362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Sending DeleteTablet for 3 replicas of tablet dd086e8ee13d471d8b9da70add941b19
I20260808 17:48:09.355305 19567 tablet_service.cc:1514] Processing DeleteTablet for tablet dd086e8ee13d471d8b9da70add941b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:09 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45816
I20260808 17:48:09.355314 19773 tablet_service.cc:1514] Processing DeleteTablet for tablet dd086e8ee13d471d8b9da70add941b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:09 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:58724
I20260808 17:48:09.355361 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 19462
I20260808 17:48:09.360846 20154 tablet_replica.cc:330] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: stopping tablet replica
I20260808 17:48:09.361013 20154 raft_consensus.cc:2229] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:09.361155 20154 raft_consensus.cc:2258] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:09.362115 20154 ts_tablet_manager.cc:1905] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:09.364063 20154 ts_tablet_manager.cc:1918] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:09.364162 20154 log.cc:1196] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/dd086e8ee13d471d8b9da70add941b19
I20260808 17:48:09.364463 20154 ts_tablet_manager.cc:1939] T dd086e8ee13d471d8b9da70add941b19 P 8e03a3df12d643e49ae99ed29c97043e: Deleting consensus metadata
I20260808 17:48:09.364944 19345 catalog_manager.cc:4841] TS 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129): tablet dd086e8ee13d471d8b9da70add941b19 (table test-table [id=294bbdeb8c6f44bc91e4a48e1f11ab0c]) successfully deleted
I20260808 17:48:09.370750 19993 tablet_service.cc:1514] Processing DeleteTablet for tablet dd086e8ee13d471d8b9da70add941b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:09 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:54336
I20260808 17:48:09.371063 20156 tablet_replica.cc:330] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: stopping tablet replica
I20260808 17:48:09.371227 20156 raft_consensus.cc:2229] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 17:48:09.371258 19344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:44639 (error 108)
I20260808 17:48:09.371363 20156 raft_consensus.cc:2258] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:09.371563 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 19679
I20260808 17:48:09.371929 20156 ts_tablet_manager.cc:1905] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:09.376634 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.375548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:44639 (local address 127.0.0.1:45830)
0808 17:48:09.376537 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.376579 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:44639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":967}
W20260808 17:48:09.376772 19344 catalog_manager.cc:4584] TS 6fbba6faf9e24bacb7f6f0b8b19cd2e1 (127.29.62.193:44639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dd086e8ee13d471d8b9da70add941b19: Network error: Client connection negotiation failed: client connection to 127.29.62.193:44639: connect: Connection refused (error 111)
I20260808 17:48:09.381665 20156 ts_tablet_manager.cc:1918] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:09.381826 20156 log.cc:1196] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/dd086e8ee13d471d8b9da70add941b19
I20260808 17:48:09.382272 20156 ts_tablet_manager.cc:1939] T dd086e8ee13d471d8b9da70add941b19 P 30cfdc793f7d409ca81ce400d68c168d: Deleting consensus metadata
I20260808 17:48:09.383127 19343 catalog_manager.cc:4841] TS 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): tablet dd086e8ee13d471d8b9da70add941b19 (table test-table [id=294bbdeb8c6f44bc91e4a48e1f11ab0c]) successfully deleted
I20260808 17:48:09.386976 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 19888
I20260808 17:48:09.399554 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 19320
I20260808 17:48:09.410323 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40337
--webserver_interface=127.29.62.254
--webserver_port=39391
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:09.495308 20158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:09.495508 20158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:09.495530 20158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:09.496709 20158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:09.496752 20158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:09.496767 20158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:09.496780 20158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:09.496793 20158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:09.498288 20158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:09.498589 20158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:09.498853 20158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:09.509282 20171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.509322 20170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.509313 20173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:09.509503 20158 server_base.cc:1034] running on GCE node
I20260808 17:48:09.509769 20158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:09.510102 20158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:09.511270 20158 hybrid_clock.cc:648] HybridClock initialized: now 1786211289511258 us; error 44 us; skew 500 ppm
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:09.513867 20158 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:48:09.514323 20158 webserver.cc:460] Webserver started at http://127.29.62.254:39391/ using document root <none> and password file <none>
I20260808 17:48:09.514611 20158 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:09.514701 20158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:09.516101 20158 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:48:09.516775 20183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.516942 20158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:48:09.517053 20158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "64541828ad6041f2bc0982bdb783e480"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "7fd07c9bdad99cab756ce6ecc7dcbb89"
server_key_iv: "362c6ba3ce9509bb55329a84110e856c"
server_key_version: "encryptionkey@0"
I20260808 17:48:09.517482 20158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:09.530766 20158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:09.532689 20158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:09.532863 20158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:09.537180 20158 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40337
I20260808 17:48:09.537217 20274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40337 every 8 connection(s)
I20260808 17:48:09.537667 20158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:48:09.539350 20275 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:48:09.540052 20275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Bootstrap starting.
I20260808 17:48:09.546566 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 20158
I20260808 17:48:09.546875 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:44639
--local_ip_for_outbound_sockets=127.29.62.193
--tserver_master_addrs=127.29.62.254:40337
--webserver_port=44337
--webserver_interface=127.29.62.193
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:09.549044 20275 log.cc:824] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:09.551542 20275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:48:09.551895 20275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Bootstrap complete.
I20260808 17:48:09.554304 20275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:09.554622 20275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64541828ad6041f2bc0982bdb783e480, State: Initialized, Role: FOLLOWER
I20260808 17:48:09.554805 20275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:09.554888 20275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:48:09.554965 20275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:48:09.555018 20275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:48:09.556048 20275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:09.556141 20275 leader_election.cc:302] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64541828ad6041f2bc0982bdb783e480; no voters:
I20260808 17:48:09.556336 20275 leader_election.cc:288] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:48:09.556403 20283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:48:09.556569 20283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 2 LEADER]: Becoming Leader. State: Replica: 64541828ad6041f2bc0982bdb783e480, State: Running, Role: LEADER
I20260808 17:48:09.556602 20275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:48:09.556707 20283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:09.557047 20283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64541828ad6041f2bc0982bdb783e480. Latest consensus state: current_term: 2 leader_uuid: "64541828ad6041f2bc0982bdb783e480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } } }
I20260808 17:48:09.557137 20283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:09.557735 20284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "64541828ad6041f2bc0982bdb783e480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } } }
I20260808 17:48:09.557971 20284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:09.558514 20303 catalog_manager.cc:1259] Loaded cluster ID: 71f93dc438744bdea73914f558733be1
I20260808 17:48:09.558570 20303 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: loading cluster ID for follower catalog manager: success
W20260808 17:48:09.558771 20303 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260808 17:48:09.558825 20303 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260808 17:48:09.558900 20292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:48:09.559394 20292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:48:09.559495 20292 catalog_manager.cc:1259] Loaded cluster ID: 71f93dc438744bdea73914f558733be1
I20260808 17:48:09.559537 20292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:48:09.559722 20292 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 17:48:09.564828 20292 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 17:48:09.567449 20292 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:48:09.567656 20292 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Loaded TSK: 0
I20260808 17:48:09.567867 20292 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:48:09.635994 20279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:09.636205 20279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:09.636230 20279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:09.637427 20279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:09.637473 20279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:09.637501 20279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:48:09.639051 20279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:44639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=44337
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:09.639308 20279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:09.639580 20279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:09.645696 20315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.645704 20314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:09.645901 20279 server_base.cc:1034] running on GCE node
W20260808 17:48:09.645717 20317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:09.646260 20279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:09.646575 20279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:09.647737 20279 hybrid_clock.cc:648] HybridClock initialized: now 1786211289647728 us; error 27 us; skew 500 ppm
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:09.650344 20279 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:48:09.650923 20279 webserver.cc:460] Webserver started at http://127.29.62.193:44337/ using document root <none> and password file <none>
I20260808 17:48:09.651187 20279 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:09.651247 20279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:09.652964 20279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:48:09.653733 20332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.653888 20279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:09.653985 20279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "92a1f708b8dc32b452e55a61aca89a86"
server_key_iv: "6d01597abac138b0cd5f588a9387c0e0"
server_key_version: "encryptionkey@0"
I20260808 17:48:09.654520 20279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:09.682651 20279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:09.685425 20279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:09.685585 20279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:09.685861 20279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:09.686348 20342 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:48:09.688225 20279 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:48:09.688301 20279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:48:09.688349 20279 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:48:09.689018 20279 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:48:09.689076 20279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 17:48:09.689131 20342 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Bootstrap starting.
I20260808 17:48:09.696442 20279 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:44639
I20260808 17:48:09.696848 20279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:48:09.697561 20522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:44639 every 8 connection(s)
I20260808 17:48:09.699811 20342 log.cc:824] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:09.701954 20342 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:48:09.702252 20342 tablet_bootstrap.cc:492] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Bootstrap complete.
I20260808 17:48:09.703269 20342 ts_tablet_manager.cc:1397] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.007s
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:09.704303 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 20279
I20260808 17:48:09.704582 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:45129
--local_ip_for_outbound_sockets=127.29.62.194
--tserver_master_addrs=127.29.62.254:40337
--webserver_port=34831
--webserver_interface=127.29.62.194
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:09.704661 20342 raft_consensus.cc:348] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.705068 20342 raft_consensus.cc:729] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fbba6faf9e24bacb7f6f0b8b19cd2e1, State: Initialized, Role: FOLLOWER
I20260808 17:48:09.705226 20342 consensus_queue.cc:260] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:09.705520 20342 ts_tablet_manager.cc:1428] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260808 17:48:09.708750 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.697589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:38139)
0808 17:48:09.697792 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.697837 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.698467 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.698930 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.698934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.699233 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:09.699433 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.699439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.700790 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.700792 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.701573 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.701575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.702226 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.702227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.702418 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.702999 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:09.703007 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:09.704773 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:09.707113 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.707121 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.707123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.707352 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.707624 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.707625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.707626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.707736 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.708128 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:09.708135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:09.708271 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.708379 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.708558 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260808 17:48:09.708750 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.697817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:38139 (local address 127.29.62.254:40337)
0808 17:48:09.698048 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.698056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.698098 (+ 42us) server_negotiation.cc:407] Connection header received
0808 17:48:09.698708 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.698713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.698756 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.698900 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:09.699560 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.700658 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.701671 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.702102 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.702230 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.704889 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:09.704893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:09.704893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:09.704906 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:09.707006 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.707450 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.707453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.707454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.707549 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.707812 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.707813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.707814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.708043 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:09.708136 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.708437 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.708564 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":119,"threads_started":1}
I20260808 17:48:09.709640 20523 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40337
I20260808 17:48:09.709730 20523 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:09.709915 20523 heartbeater.cc:511] Master 127.29.62.254:40337 requested a full tablet report, sending...
I20260808 17:48:09.710697 20213 ts_manager.cc:194] Registered new tserver with Master: 6fbba6faf9e24bacb7f6f0b8b19cd2e1 (127.29.62.193:44639)
I20260808 17:48:09.711887 20213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:38139
I20260808 17:48:09.712817 20523 heartbeater.cc:503] Master 127.29.62.254:40337 was elected leader, sending a full tablet report...
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.193@KRBTEST.COM
I20260808 17:48:09.720206 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.711283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:44639 (local address 127.0.0.1:45834)
0808 17:48:09.711489 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.711518 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.711579 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.711805 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.711814 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.712197 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:09.712264 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.712279 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.713439 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.713441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.714474 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.714478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.714947 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.714948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.715040 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.715493 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:09.715499 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:09.717190 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:09.718966 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.718973 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.718976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.719233 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.719497 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.719498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.719498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.719567 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.719832 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:09.719838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:09.719889 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.719979 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.720096 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":91,"threads_started":1}
I20260808 17:48:09.720136 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.711382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.29.62.193:44639)
0808 17:48:09.711559 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.711563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.711575 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:48:09.711689 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.711692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.711707 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.711770 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:09.712371 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.713357 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.714565 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.714859 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.714917 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.717280 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:09.717287 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:09.717287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:09.717304 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:09.718887 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.719358 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.719362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.719362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.719428 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.719632 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.719632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.719633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.719766 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:09.719840 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.720002 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.720074 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260808 17:48:09.720670 20396 tablet_service.cc:1514] Processing DeleteTablet for tablet dd086e8ee13d471d8b9da70add941b19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:09 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45834
I20260808 17:48:09.721205 20536 tablet_replica.cc:330] stopping tablet replica
I20260808 17:48:09.721323 20536 raft_consensus.cc:2229] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:09.721400 20536 raft_consensus.cc:2258] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:09.721740 20536 ts_tablet_manager.cc:1905] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:09.724077 20536 ts_tablet_manager.cc:1918] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:09.724162 20536 log.cc:1196] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/dd086e8ee13d471d8b9da70add941b19
I20260808 17:48:09.724416 20536 ts_tablet_manager.cc:1939] T dd086e8ee13d471d8b9da70add941b19 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Deleting consensus metadata
I20260808 17:48:09.724972 20189 catalog_manager.cc:4841] TS 6fbba6faf9e24bacb7f6f0b8b19cd2e1 (127.29.62.193:44639): tablet dd086e8ee13d471d8b9da70add941b19 (table test-table [id=294bbdeb8c6f44bc91e4a48e1f11ab0c]) successfully deleted
W20260808 17:48:09.793882 20532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:09.794094 20532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:09.794119 20532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:09.795238 20532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:09.795279 20532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:09.795303 20532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:48:09.796681 20532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:45129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=34831
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:09.796871 20532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:09.797051 20532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:09.802326 20544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.802340 20549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.802395 20543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:09.802583 20532 server_base.cc:1034] running on GCE node
I20260808 17:48:09.802800 20532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:09.803086 20532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:09.804248 20532 hybrid_clock.cc:648] HybridClock initialized: now 1786211289804240 us; error 33 us; skew 500 ppm
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:09.806774 20532 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:48:09.807224 20532 webserver.cc:460] Webserver started at http://127.29.62.194:34831/ using document root <none> and password file <none>
I20260808 17:48:09.807412 20532 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:09.807464 20532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:09.808799 20532 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:09.809476 20566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.809687 20532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:09.809796 20532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "8e03a3df12d643e49ae99ed29c97043e"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "484532d4d07984cd48011d8f0b7e220f"
server_key_iv: "fa92e4629b06b105523ea3cac61881f4"
server_key_version: "encryptionkey@0"
I20260808 17:48:09.810194 20532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:09.833902 20532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:09.835867 20532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:09.836050 20532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:09.836300 20532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:09.836653 20532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:09.836709 20532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.836746 20532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:09.836794 20532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.843181 20532 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:45129
I20260808 17:48:09.843225 20743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:45129 every 8 connection(s)
I20260808 17:48:09.843585 20532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:09.851423 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 20532
I20260808 17:48:09.851694 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:45017
--local_ip_for_outbound_sockets=127.29.62.195
--tserver_master_addrs=127.29.62.254:40337
--webserver_port=45685
--webserver_interface=127.29.62.195
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:09.853484 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.844359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:56045 (local address 127.29.62.254:40337)
0808 17:48:09.844431 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.844433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.844668 (+ 235us) server_negotiation.cc:407] Connection header received
0808 17:48:09.845201 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.845202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.845217 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.845310 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:09.846451 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.846972 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.847969 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.848191 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.848249 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.850343 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:09.850344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:09.850344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:09.850351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:09.851978 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.852468 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.852469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.852469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.852522 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.852795 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.852796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.852796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.852899 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:09.852949 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.853327 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.853407 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:48:09.853852 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.844370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:56045)
0808 17:48:09.844559 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.844595 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.845085 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.845425 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.845431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.846104 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:09.846300 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.846304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.847064 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.847065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.847877 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.847879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.848271 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.848271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.848373 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.848871 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:09.848879 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:09.850245 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:09.852052 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.852060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.852062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.852384 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.852585 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.852586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.852586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.852732 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.852964 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:09.852986 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:09.853158 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.853277 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.853461 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260808 17:48:09.854348 20745 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40337
I20260808 17:48:09.854494 20745 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:09.854688 20745 heartbeater.cc:511] Master 127.29.62.254:40337 requested a full tablet report, sending...
I20260808 17:48:09.855094 20213 ts_manager.cc:194] Registered new tserver with Master: 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129)
I20260808 17:48:09.855598 20213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:56045
W20260808 17:48:09.936431 20750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:09.936636 20750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:09.936659 20750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:09.937718 20750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:09.937752 20750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:09.937778 20750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:48:09.939316 20750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:45017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=45685
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:09.939525 20750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:09.939702 20750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:09.944708 20758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.944708 20761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:09.944797 20757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:09.944864 20750 server_base.cc:1034] running on GCE node
I20260808 17:48:09.945132 20750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:09.945346 20750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:09.946439 20750 hybrid_clock.cc:648] HybridClock initialized: now 1786211289946426 us; error 31 us; skew 500 ppm
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:09.949061 20750 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:48:09.949556 20750 webserver.cc:460] Webserver started at http://127.29.62.195:45685/ using document root <none> and password file <none>
I20260808 17:48:09.949811 20750 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:09.949869 20750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:09.951314 20750 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:09.952155 20772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.952379 20750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:09.952494 20750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "30cfdc793f7d409ca81ce400d68c168d"
format_stamp: "Formatted at 2026-08-08 17:48:09 on dist-test-slave-tk9z"
server_key: "2db4c8412e9cffa7a1a217998bd4e816"
server_key_iv: "a81dbe754fbbb2aebfc9af904e382817"
server_key_version: "encryptionkey@0"
I20260808 17:48:09.952855 20750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:09.967643 20750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:09.969741 20750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:09.969919 20750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:09.970186 20750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:09.970589 20750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:09.970646 20750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.970683 20750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:09.970727 20750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:09.976694 20750 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:45017
I20260808 17:48:09.976737 20947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:45017 every 8 connection(s)
I20260808 17:48:09.977149 20750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:48:09.978648 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 20750
Aug 08 17:48:09 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211289, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:09.985942 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.979079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.29.62.254:40337)
0808 17:48:09.979179 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.979181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.979331 (+ 150us) server_negotiation.cc:407] Connection header received
0808 17:48:09.979414 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.979415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.979427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.979484 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:09.980155 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.980729 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.981484 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.981760 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.981812 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.983147 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:09.983148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:09.983149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:09.983157 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:09.984658 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.984984 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.984985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.984985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.985048 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.985294 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.985295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.985296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.985441 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:09.985636 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.985677 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.985815 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":81,"thread_start_us":60,"threads_started":1}
I20260808 17:48:09.987550 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.977929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:34941 (local address 127.29.62.254:40337)
0808 17:48:09.977982 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.977985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.978204 (+ 219us) server_negotiation.cc:407] Connection header received
0808 17:48:09.978813 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.978815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.978829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.978898 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:09.979923 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.980505 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.981611 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.981816 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.981882 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.984319 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:09.984320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:09.984321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:09.984328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:09.985952 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.986583 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.986584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.986584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.986644 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:09.986892 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:09.986892 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:09.986893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:09.987002 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:09.987090 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:09.987374 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 17:48:09.987456 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":21}
I20260808 17:48:09.987769 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.977930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:34941)
0808 17:48:09.978087 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:09.978121 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:09.978677 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:09.978983 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:09.978988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:09.979637 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:09.979832 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.979837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.980600 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.980602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.981529 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:09.981531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.981912 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.981913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:09.982019 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:09.982598 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:09.982606 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:09.984219 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:09.986043 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.986051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.986054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.986504 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.986698 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:09.986699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:09.986699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:09.986816 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:09.987086 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:09.987094 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:09.987226 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 17:48:09.987340 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 17:48:09.987516 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260808 17:48:09.988200 20948 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40337
I20260808 17:48:09.988300 20948 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:09.988436 20948 heartbeater.cc:511] Master 127.29.62.254:40337 requested a full tablet report, sending...
I20260808 17:48:09.988776 20213 ts_manager.cc:194] Registered new tserver with Master: 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
I20260808 17:48:09.989100 20213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:34941
I20260808 17:48:09.997525 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:48:10.000659 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:09.997968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.29.62.254:40337)
0808 17:48:09.998042 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:09.998044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:09.998144 (+ 100us) server_negotiation.cc:407] Connection header received
0808 17:48:09.998239 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:09.998239 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:09.998288 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:09.998362 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:48:09.998559 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:09.999064 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:09.999840 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.000061 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.000140 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.000331 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:48:10.000434 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:48:10.000534 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:10.000557 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 17:48:10.000577 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:48:10.001101 20213 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:48:10.002789 20213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:48:10.005308 20396 tablet_service.cc:1467] Processing CreateTablet for tablet 892cc958836744c0ad281efca839b2f3 (DEFAULT_TABLE table=test-table [id=9288429635d24c02a21d223619308f52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:10.005558 20396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892cc958836744c0ad281efca839b2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:10.007385 20342 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Bootstrap starting.
I20260808 17:48:10.007740 20342 tablet_bootstrap.cc:654] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:10.008469 20342 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: No bootstrap required, opened a new log
I20260808 17:48:10.008545 20342 ts_tablet_manager.cc:1397] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 17:48:10.008925 20342 raft_consensus.cc:348] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.009030 20342 raft_consensus.cc:374] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:10.009080 20342 raft_consensus.cc:729] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fbba6faf9e24bacb7f6f0b8b19cd2e1, State: Initialized, Role: FOLLOWER
I20260808 17:48:10.009141 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.004969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45129 (local address 127.0.0.1:58726)
0808 17:48:10.005024 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.005035 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:10.005082 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:10.005691 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:10.005692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:10.005694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:10.005741 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:10.005743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.006864 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.006865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:10.007812 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:10.007816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.008838 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.008839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:10.008911 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.008937 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 17:48:10.009015 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:48:10.009066 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:48:10.009127 20342 consensus_queue.cc:260] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.009183 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.005224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.29.62.194:45129)
0808 17:48:10.005482 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:10.005493 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:10.005502 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:48:10.005535 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:10.005537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:10.005550 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:10.005816 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:10.005889 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.006758 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.007916 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.008730 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.008824 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.008895 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:10.009043 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 17:48:10.009091 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":124,"threads_started":1}
I20260808 17:48:10.009239 20342 ts_tablet_manager.cc:1428] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 17:48:10.010280 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.005050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45017 (local address 127.0.0.1:54338)
0808 17:48:10.005214 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.005225 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:10.005277 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:10.005766 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:10.005767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:10.005770 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:10.005825 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:10.005827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.007422 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.007423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:10.008674 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:10.008677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.010010 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.010011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:10.010075 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.010080 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:48:10.010136 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 17:48:10.010201 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260808 17:48:10.010179 20631 tablet_service.cc:1467] Processing CreateTablet for tablet 892cc958836744c0ad281efca839b2f3 (DEFAULT_TABLE table=test-table [id=9288429635d24c02a21d223619308f52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:10.010622 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.005295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.29.62.195:45017)
0808 17:48:10.005497 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:10.005502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:10.005515 (+ 13us) server_negotiation.cc:407] Connection header received
0808 17:48:10.005563 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:10.005565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:10.005583 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:10.005974 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:10.006051 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.007277 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.008834 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.009912 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.010010 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.010068 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:10.010459 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 17:48:10.010516 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":89,"threads_started":1}
I20260808 17:48:10.010680 20631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892cc958836744c0ad281efca839b2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:10.010887 20531 raft_consensus.cc:482] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:10.011005 20531 raft_consensus.cc:504] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.011365 20531 leader_election.cc:288] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129), 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
I20260808 17:48:10.013785 20968 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: Bootstrap starting.
I20260808 17:48:10.014578 20968 tablet_bootstrap.cc:654] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:10.014506 20846 tablet_service.cc:1467] Processing CreateTablet for tablet 892cc958836744c0ad281efca839b2f3 (DEFAULT_TABLE table=test-table [id=9288429635d24c02a21d223619308f52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:10.014871 20846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892cc958836744c0ad281efca839b2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:10.015177 20968 log.cc:824] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:10.015928 20968 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: No bootstrap required, opened a new log
I20260808 17:48:10.015990 20968 ts_tablet_manager.cc:1397] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:48:10.017562 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.011511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45017 (local address 127.29.62.193:52193)
0808 17:48:10.014073 (+ 2562us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.014083 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:10.014145 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:10.014328 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:10.014329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:10.014331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:10.014432 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:10.014434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.015302 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.015303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:10.016499 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:10.016502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.017222 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.017223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:10.017311 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.017335 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 17:48:10.017386 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:48:10.017431 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2547,"thread_start_us":86,"threads_started":1}
I20260808 17:48:10.017716 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.014230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:40741 (local address 127.29.62.194:45129)
0808 17:48:10.014541 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:10.014543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:10.014550 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:48:10.014569 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:10.014570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:10.014579 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:10.014625 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:10.014812 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.015461 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.016702 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.017262 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.017319 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.017327 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:10.017597 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 17:48:10.017663 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260808 17:48:10.017777 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.011690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:52193 (local address 127.29.62.195:45017)
0808 17:48:10.011737 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:10.011740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:10.014133 (+ 2393us) server_negotiation.cc:407] Connection header received
0808 17:48:10.014195 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:10.014195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:10.014205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:10.014267 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:10.014524 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.015216 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.016595 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.017116 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.017180 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.017189 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:10.017674 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 17:48:10.017710 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:48:10.017867 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.011482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45129 (local address 127.29.62.193:40741)
0808 17:48:10.011651 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.011662 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:10.011718 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:10.014664 (+ 2946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:10.014665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:10.014667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:10.014745 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:10.014747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.015542 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.015543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:10.016620 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:10.016622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.017369 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.017370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:10.017436 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.017439 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:48:10.017483 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 17:48:10.017516 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260808 17:48:10.018172 20666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892cc958836744c0ad281efca839b2f3" candidate_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e03a3df12d643e49ae99ed29c97043e" is_pre_election: true
I20260808 17:48:10.018179 20968 raft_consensus.cc:348] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.018556 20968 raft_consensus.cc:374] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:10.018607 20968 raft_consensus.cc:729] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e03a3df12d643e49ae99ed29c97043e, State: Initialized, Role: FOLLOWER
W20260808 17:48:10.018582 20334 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
I20260808 17:48:10.018707 20968 consensus_queue.cc:260] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.018858 20666 raft_consensus.cc:2454] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fbba6faf9e24bacb7f6f0b8b19cd2e1 in term 0.
I20260808 17:48:10.019030 20968 ts_tablet_manager.cc:1428] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:48:10.019053 20745 heartbeater.cc:503] Master 127.29.62.254:40337 was elected leader, sending a full tablet report...
I20260808 17:48:10.019088 20338 leader_election.cc:302] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fbba6faf9e24bacb7f6f0b8b19cd2e1, 8e03a3df12d643e49ae99ed29c97043e; no voters: 30cfdc793f7d409ca81ce400d68c168d
I20260808 17:48:10.019198 20531 raft_consensus.cc:2781] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:48:10.019243 20531 raft_consensus.cc:482] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:48:10.019266 20531 raft_consensus.cc:3037] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:10.019696 20974 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: Bootstrap starting.
I20260808 17:48:10.020139 20531 raft_consensus.cc:504] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.020319 20531 leader_election.cc:288] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 election: Requested vote from peers 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129), 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
I20260808 17:48:10.020430 20666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892cc958836744c0ad281efca839b2f3" candidate_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e03a3df12d643e49ae99ed29c97043e"
I20260808 17:48:10.020467 20974 tablet_bootstrap.cc:654] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:10.020509 20666 raft_consensus.cc:3037] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:10.021111 20974 log.cc:824] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:10.021363 20666 raft_consensus.cc:2454] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fbba6faf9e24bacb7f6f0b8b19cd2e1 in term 1.
I20260808 17:48:10.020537 20882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892cc958836744c0ad281efca839b2f3" candidate_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30cfdc793f7d409ca81ce400d68c168d"
W20260808 17:48:10.021675 20334 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): Illegal state: must be running to vote when last-logged opid is not known
I20260808 17:48:10.021756 20338 leader_election.cc:302] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fbba6faf9e24bacb7f6f0b8b19cd2e1, 8e03a3df12d643e49ae99ed29c97043e; no voters: 30cfdc793f7d409ca81ce400d68c168d
I20260808 17:48:10.021876 20531 raft_consensus.cc:2781] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:10.022033 20974 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: No bootstrap required, opened a new log
I20260808 17:48:10.022032 20531 raft_consensus.cc:686] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 LEADER]: Becoming Leader. State: Replica: 6fbba6faf9e24bacb7f6f0b8b19cd2e1, State: Running, Role: LEADER
I20260808 17:48:10.022099 20974 ts_tablet_manager.cc:1397] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:48:10.022104 20531 consensus_queue.cc:237] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.022825 20213 catalog_manager.cc:5495] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fbba6faf9e24bacb7f6f0b8b19cd2e1 (127.29.62.193). New cstate: current_term: 1 leader_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } health_report { overall_health: HEALTHY } } }
I20260808 17:48:10.023437 20974 raft_consensus.cc:348] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.023622 20974 raft_consensus.cc:374] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:10.023672 20974 raft_consensus.cc:729] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30cfdc793f7d409ca81ce400d68c168d, State: Initialized, Role: FOLLOWER
I20260808 17:48:10.023795 20974 consensus_queue.cc:260] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.024063 20974 ts_tablet_manager.cc:1428] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:48:10.024086 20948 heartbeater.cc:503] Master 127.29.62.254:40337 was elected leader, sending a full tablet report...
I20260808 17:48:10.030913 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.028534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.29.62.193:44639)
0808 17:48:10.028578 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:10.028581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:10.028592 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:48:10.028633 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:10.028634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:10.028651 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:10.028735 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:48:10.028836 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.029330 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.030070 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:10.030302 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:10.030379 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:10.030584 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:48:10.030679 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:48:10.030761 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:10.030812 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 17:48:10.030835 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:48:10.032207 20882 raft_consensus.cc:3037] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:10.032917 20882 raft_consensus.cc:1261] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d [term 1 FOLLOWER]: Refusing update from remote peer 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:10.033254 20666 raft_consensus.cc:1261] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 1 FOLLOWER]: Refusing update from remote peer 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:10.033258 20531 consensus_queue.cc:1035] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:10.033612 20531 consensus_queue.cc:1035] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:10.035789 20980 mvcc.cc:205] Tried to move back new op lower bound from 7316321443969884160 to 7316321443931152384. Current Snapshot: MvccSnapshot[applied={T|T < 7316321443969884160}]
I20260808 17:48:10.036275 20982 mvcc.cc:205] Tried to move back new op lower bound from 7316321443969884160 to 7316321443931152384. Current Snapshot: MvccSnapshot[applied={T|T < 7316321443969884160}]
I20260808 17:48:10.038055 20213 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:10.038161 20213 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:10.038971 20213 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Sending DeleteTablet for 3 replicas of tablet 892cc958836744c0ad281efca839b2f3
I20260808 17:48:10.039362 20631 tablet_service.cc:1514] Processing DeleteTablet for tablet 892cc958836744c0ad281efca839b2f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:10 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:58726
I20260808 17:48:10.039393 20846 tablet_service.cc:1514] Processing DeleteTablet for tablet 892cc958836744c0ad281efca839b2f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:10 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:54338
I20260808 17:48:10.039635 20989 tablet_replica.cc:330] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: stopping tablet replica
I20260808 17:48:10.039644 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 20279
I20260808 17:48:10.039762 20989 raft_consensus.cc:2229] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:10.039872 20989 raft_consensus.cc:2258] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:10.040011 20990 tablet_replica.cc:330] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: stopping tablet replica
I20260808 17:48:10.040086 20990 raft_consensus.cc:2229] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:10.044390 20990 raft_consensus.cc:2258] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:10.045969 20990 ts_tablet_manager.cc:1905] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:10.048096 20989 ts_tablet_manager.cc:1905] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:10.048123 20990 ts_tablet_manager.cc:1918] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:10.048197 20990 log.cc:1196] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/892cc958836744c0ad281efca839b2f3
I20260808 17:48:10.048580 20990 ts_tablet_manager.cc:1939] T 892cc958836744c0ad281efca839b2f3 P 8e03a3df12d643e49ae99ed29c97043e: Deleting consensus metadata
I20260808 17:48:10.049144 20190 catalog_manager.cc:4841] TS 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129): tablet 892cc958836744c0ad281efca839b2f3 (table test-table [id=9288429635d24c02a21d223619308f52]) successfully deleted
I20260808 17:48:10.049823 20989 ts_tablet_manager.cc:1918] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:10.049887 20989 log.cc:1196] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/892cc958836744c0ad281efca839b2f3
I20260808 17:48:10.050292 20989 ts_tablet_manager.cc:1939] T 892cc958836744c0ad281efca839b2f3 P 30cfdc793f7d409ca81ce400d68c168d: Deleting consensus metadata
I20260808 17:48:10.050928 20186 catalog_manager.cc:4841] TS 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): tablet 892cc958836744c0ad281efca839b2f3 (table test-table [id=9288429635d24c02a21d223619308f52]) successfully deleted
W20260808 17:48:10.059507 20189 connection.cc:535] client connection to 127.29.62.193:44639 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 17:48:10.059623 20189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:48:10.060001 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 20532
I20260808 17:48:10.060141 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.059985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:44639 (local address 127.0.0.1:45850)
0808 17:48:10.060043 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.060100 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:44639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 17:48:10.064154 20189 catalog_manager.cc:4584] TS 6fbba6faf9e24bacb7f6f0b8b19cd2e1 (127.29.62.193:44639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 892cc958836744c0ad281efca839b2f3: Network error: Client connection negotiation failed: client connection to 127.29.62.193:44639: connect: Connection refused (error 111)
I20260808 17:48:10.071470 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 20750
I20260808 17:48:10.080112 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 20158
I20260808 17:48:10.089622 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40337
--webserver_interface=127.29.62.254
--webserver_port=39391
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:10.173756 20994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:10.173976 20994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:10.174000 20994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:10.175215 20994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:10.175263 20994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:10.175278 20994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:10.175336 20994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:10.175362 20994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:10.176856 20994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=39391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:10.177122 20994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:10.177373 20994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:10.182157 21003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:10.182142 21005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:10.182184 21002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:10.182611 20994 server_base.cc:1034] running on GCE node
I20260808 17:48:10.182868 20994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:10.183167 20994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:10.184367 20994 hybrid_clock.cc:648] HybridClock initialized: now 1786211290184360 us; error 35 us; skew 500 ppm
Aug 08 17:48:10 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211290, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:10.187036 20994 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:48:10.187569 20994 webserver.cc:460] Webserver started at http://127.29.62.254:39391/ using document root <none> and password file <none>
I20260808 17:48:10.187813 20994 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:10.187891 20994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:10.189278 20994 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 17:48:10.189947 21013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.190136 20994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.190272 20994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "64541828ad6041f2bc0982bdb783e480"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "7fd07c9bdad99cab756ce6ecc7dcbb89"
server_key_iv: "362c6ba3ce9509bb55329a84110e856c"
server_key_version: "encryptionkey@0"
I20260808 17:48:10.190704 20994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:10.206113 20994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:10.208437 20994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:10.208611 20994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:10.213040 20994 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40337
I20260808 17:48:10.213086 21097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40337 every 8 connection(s)
I20260808 17:48:10.213454 20994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:48:10.215018 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 20994
I20260808 17:48:10.215190 21098 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:48:10.215350 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:44639
--local_ip_for_outbound_sockets=127.29.62.193
--tserver_master_addrs=127.29.62.254:40337
--webserver_port=44337
--webserver_interface=127.29.62.193
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:10.216285 21098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Bootstrap starting.
I20260808 17:48:10.225188 21098 log.cc:824] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:10.227463 21098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:48:10.227789 21098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480: Bootstrap complete.
I20260808 17:48:10.230178 21098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:10.230477 21098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64541828ad6041f2bc0982bdb783e480, State: Initialized, Role: FOLLOWER
I20260808 17:48:10.230664 21098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:10.230754 21098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:48:10.230803 21098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:48:10.230850 21098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 2 FOLLOWER]: Advancing to term 3
I20260808 17:48:10.231832 21098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:10.231947 21098 leader_election.cc:302] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64541828ad6041f2bc0982bdb783e480; no voters:
I20260808 17:48:10.232139 21098 leader_election.cc:288] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 17:48:10.232177 21106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 17:48:10.232295 21106 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [term 3 LEADER]: Becoming Leader. State: Replica: 64541828ad6041f2bc0982bdb783e480, State: Running, Role: LEADER
I20260808 17:48:10.232393 21106 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } }
I20260808 17:48:10.232417 21098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:48:10.232697 21110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64541828ad6041f2bc0982bdb783e480. Latest consensus state: current_term: 3 leader_uuid: "64541828ad6041f2bc0982bdb783e480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } } }
I20260808 17:48:10.232785 21110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:10.232897 21109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "64541828ad6041f2bc0982bdb783e480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64541828ad6041f2bc0982bdb783e480" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40337 } } }
I20260808 17:48:10.233018 21109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64541828ad6041f2bc0982bdb783e480 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:10.233028 21112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:48:10.234411 21112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:48:10.234532 21112 catalog_manager.cc:1259] Loaded cluster ID: 71f93dc438744bdea73914f558733be1
I20260808 17:48:10.234571 21112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 17:48:10.235093 21112 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1d6eacd9ad google::LogMessage::Fail() at ??:0
@ 0x7f1d6ead088a google::LogMessage::SendToLog() at ??:0
@ 0x7f1d6eacd4de google::LogMessage::Flush() at ??:0
@ 0x7f1d6eacea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1d769f20f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f1d769fb8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1d7099b0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1d70990954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1d717f46db start_thread at ??:0
@ 0x7f1d6d5cb61f clone at ??:0
W20260808 17:48:10.312469 21101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:10.312695 21101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:10.312732 21101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:10.313876 21101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:10.313942 21101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:10.313988 21101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:48:10.315544 21101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:44639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=44337
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:10.315837 21101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:10.316071 21101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:10.321213 21154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:10.321209 21150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:10.321496 21151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:10.321514 21101 server_base.cc:1034] running on GCE node
I20260808 17:48:10.321811 21101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:10.322093 21101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:10.323273 21101 hybrid_clock.cc:648] HybridClock initialized: now 1786211290323252 us; error 33 us; skew 500 ppm
Aug 08 17:48:10 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211290, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:10.326521 21101 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:48:10.326997 21101 webserver.cc:460] Webserver started at http://127.29.62.193:44337/ using document root <none> and password file <none>
I20260808 17:48:10.327253 21101 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:10.327333 21101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:10.328819 21101 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:48:10.329582 21165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.329769 21101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:10.329885 21101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "92a1f708b8dc32b452e55a61aca89a86"
server_key_iv: "6d01597abac138b0cd5f588a9387c0e0"
server_key_version: "encryptionkey@0"
I20260808 17:48:10.330229 21101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:10.356248 21101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:10.358732 21101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:10.358911 21101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:10.359165 21101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:10.359635 21177 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:48:10.361634 21101 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:48:10.361699 21101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:48:10.361743 21101 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:48:10.362468 21101 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:48:10.362527 21101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:10.362576 21177 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Bootstrap starting.
I20260808 17:48:10.370026 21101 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:44639
I20260808 17:48:10.370137 21335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:44639 every 8 connection(s)
I20260808 17:48:10.370543 21101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:48:10.371889 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 21101
I20260808 17:48:10.372201 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:45129
--local_ip_for_outbound_sockets=127.29.62.194
--tserver_master_addrs=127.29.62.254:40337
--webserver_port=34831
--webserver_interface=127.29.62.194
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:10.374418 21177 log.cc:824] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:10.377301 21177 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:48:10.377656 21177 tablet_bootstrap.cc:492] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Bootstrap complete.
I20260808 17:48:10.379001 21177 ts_tablet_manager.cc:1397] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.013s
I20260808 17:48:10.380488 21177 raft_consensus.cc:348] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.380852 21177 raft_consensus.cc:729] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fbba6faf9e24bacb7f6f0b8b19cd2e1, State: Initialized, Role: FOLLOWER
I20260808 17:48:10.381026 21177 consensus_queue.cc:260] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:10.381340 21177 ts_tablet_manager.cc:1428] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260808 17:48:10.433647 21339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:48:10.371250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:56755)
0808 17:48:10.371454 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.371490 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:10.372120 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:10.433410 (+ 61290us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
W20260808 17:48:10.433938 21336 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:40337 (0 consecutive failures): Network error: Failed to ping master at 127.29.62.254:40337: Client connection negotiation failed: client connection to 127.29.62.254:40337: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:48:10.434309 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.434191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:45181)
0808 17:48:10.434243 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.434287 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:48:10.434675 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.434589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:35203)
0808 17:48:10.434633 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.434657 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:48:10.434810 21336 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:48:10.461913 21340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:10.462121 21340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:10.462144 21340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:10.463330 21340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:10.463374 21340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:10.463402 21340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:48:10.464802 21340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:45129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=34831
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:10.464993 21340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:10.465179 21340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:10.470101 21352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:10.470090 21354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:10.470103 21351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:10.470275 21340 server_base.cc:1034] running on GCE node
I20260808 17:48:10.470562 21340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:10.470809 21340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:10.471984 21340 hybrid_clock.cc:648] HybridClock initialized: now 1786211290471971 us; error 25 us; skew 500 ppm
Aug 08 17:48:10 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211290, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:10.474637 21340 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:48:10.475236 21340 webserver.cc:460] Webserver started at http://127.29.62.194:34831/ using document root <none> and password file <none>
I20260808 17:48:10.475553 21340 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:10.475641 21340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:10.477025 21340 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:10.477666 21363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.477866 21340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:10.477969 21340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "8e03a3df12d643e49ae99ed29c97043e"
format_stamp: "Formatted at 2026-08-08 17:48:08 on dist-test-slave-tk9z"
server_key: "484532d4d07984cd48011d8f0b7e220f"
server_key_iv: "fa92e4629b06b105523ea3cac61881f4"
server_key_version: "encryptionkey@0"
I20260808 17:48:10.478327 21340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:10.500048 21340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:10.502615 21340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:10.502784 21340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:10.503036 21340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:10.503364 21340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:10.503422 21340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.503465 21340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:10.503494 21340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.509652 21340 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:45129
I20260808 17:48:10.509712 21540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:45129 every 8 connection(s)
I20260808 17:48:10.510056 21340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:48:10.511385 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.510758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:59609)
0808 17:48:10.511005 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.511123 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":148,"threads_started":1}
W20260808 17:48:10.511922 21541 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:40337 (0 consecutive failures): Network error: Failed to ping master at 127.29.62.254:40337: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
I20260808 17:48:10.512270 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.512156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:44485)
0808 17:48:10.512211 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.512246 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:48:10.512697 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.512587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:55983)
0808 17:48:10.512646 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.512675 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 17:48:10.512835 21541 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:48:10.519443 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 21340
I20260808 17:48:10.519722 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:45017
--local_ip_for_outbound_sockets=127.29.62.195
--tserver_master_addrs=127.29.62.254:40337
--webserver_port=45685
--webserver_interface=127.29.62.195
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:10.609836 21546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:10.610056 21546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:10.610080 21546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:10.611266 21546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:10.611307 21546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:10.611333 21546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:48:10.612964 21546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:45017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=45685
--tserver_master_addrs=127.29.62.254:40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:10.613231 21546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:10.613503 21546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:48:10.618677 21546 server_base.cc:1034] running on GCE node
W20260808 17:48:10.618631 21556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:10.618656 21553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:10.618644 21552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:10.619091 21546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:10.619349 21546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:10.620527 21546 hybrid_clock.cc:648] HybridClock initialized: now 1786211290620523 us; error 35 us; skew 500 ppm
Aug 08 17:48:10 dist-test-slave-tk9z krb5kdc[19299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211290, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:10.623189 21546 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:48:10.623662 21546 webserver.cc:460] Webserver started at http://127.29.62.195:45685/ using document root <none> and password file <none>
I20260808 17:48:10.623893 21546 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:10.623965 21546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:10.625417 21546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:10.626046 21567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.626219 21546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:48:10.626353 21546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "30cfdc793f7d409ca81ce400d68c168d"
format_stamp: "Formatted at 2026-08-08 17:48:09 on dist-test-slave-tk9z"
server_key: "2db4c8412e9cffa7a1a217998bd4e816"
server_key_iv: "a81dbe754fbbb2aebfc9af904e382817"
server_key_version: "encryptionkey@0"
I20260808 17:48:10.626782 21546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:10.655994 21546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:10.658043 21546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:10.658241 21546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:10.658574 21546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:10.658938 21546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:10.658995 21546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.659031 21546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:10.659063 21546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:10.664987 21546 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:45017
I20260808 17:48:10.665045 21736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:45017 every 8 connection(s)
I20260808 17:48:10.665431 21546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:48:10.666121 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 21546
I20260808 17:48:10.666638 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.666146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:32939)
0808 17:48:10.666338 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.666455 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
W20260808 17:48:10.667086 21737 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:40337 (0 consecutive failures): Network error: Failed to ping master at 127.29.62.254:40337: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
I20260808 17:48:10.667377 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.667288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:55125)
0808 17:48:10.667335 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.667360 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:48:10.667626 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:10.667572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:56099)
0808 17:48:10.667598 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:10.667615 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 17:48:10.667728 21737 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:48:11.435379 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:11.435143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:55319)
0808 17:48:11.435271 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:11.435309 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260808 17:48:11.513339 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:11.513138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:53429)
0808 17:48:11.513249 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:11.513283 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260808 17:48:11.538259 21748 raft_consensus.cc:482] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:11.538486 21748 raft_consensus.cc:504] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:11.538853 21748 leader_election.cc:288] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129), 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
Aug 08 17:48:11 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211290, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.195@KRBTEST.COM
Aug 08 17:48:11 dist-test-slave-tk9z krb5kdc[19299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211290, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.194@KRBTEST.COM
I20260808 17:48:11.550406 21752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:11.539134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:55737 (local address 127.29.62.194:45129)
0808 17:48:11.539308 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:11.539326 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:11.539365 (+ 39us) server_negotiation.cc:407] Connection header received
0808 17:48:11.539862 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:11.539864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:11.539903 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:11.540064 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:11.540870 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:11.541742 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:11.542832 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:11.543381 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:11.543524 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:11.546168 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:11.546172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:11.546173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:11.546185 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:11.548176 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:11.548844 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:11.548847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:11.548848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:11.549018 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:11.549340 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:11.549341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:11.549341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:11.549663 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:11.549753 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:11.550114 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 17:48:11.550268 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260808 17:48:11.550406 21753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:11.539190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:55845 (local address 127.29.62.195:45017)
0808 17:48:11.539334 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:11.539342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:11.539384 (+ 42us) server_negotiation.cc:407] Connection header received
0808 17:48:11.539840 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:11.539842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:11.539901 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:11.540045 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:11.540886 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:11.541762 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:11.542817 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:11.543321 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:11.543452 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:11.545959 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:11.545965 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:11.545965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:11.545979 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:11.548176 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:11.548844 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:11.548847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:11.548848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:11.548980 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:11.549340 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:11.549340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:11.549341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:11.549663 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:11.549753 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:11.550114 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 17:48:11.550268 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260808 17:48:11.550417 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:11.539002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45129 (local address 127.29.62.193:55737)
0808 17:48:11.539063 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:11.539074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:11.539131 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:11.540095 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:11.540098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:11.540520 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:11.540715 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:11.540722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:11.541855 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:11.541856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:11.542723 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:11.542726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:11.543512 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:11.543513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:11.543579 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:11.544204 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:11.544208 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:11.546061 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:11.548272 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:11.548282 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:11.548284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:11.548750 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:11.549093 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:11.549093 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:11.549094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:11.549261 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:11.549745 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:11.549753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:11.549915 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 17:48:11.550037 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 17:48:11.550260 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":52}
I20260808 17:48:11.550416 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:11.539032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:45017 (local address 127.29.62.193:55845)
0808 17:48:11.539284 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:11.539295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:11.539347 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:11.540157 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:11.540158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:11.540550 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:11.540716 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:11.540722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:11.541881 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:11.541882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:11.542725 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:11.542728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:11.543420 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:11.543422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:11.543541 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:11.544175 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:11.544183 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:11.545801 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:11.548272 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:11.548282 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:11.548284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:11.548750 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:11.549093 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:11.549093 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:11.549094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:11.549261 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:11.549745 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:11.549753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:11.549915 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 17:48:11.550037 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 17:48:11.550314 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":177,"thread_start_us":60,"threads_started":1}
W20260808 17:48:11.551533 21171 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
W20260808 17:48:11.551749 21169 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
I20260808 17:48:11.551802 21169 leader_election.cc:302] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fbba6faf9e24bacb7f6f0b8b19cd2e1; no voters: 30cfdc793f7d409ca81ce400d68c168d, 8e03a3df12d643e49ae99ed29c97043e
I20260808 17:48:11.551961 21748 raft_consensus.cc:2726] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:48:11.668269 21754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:11.668049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:53623)
0808 17:48:11.668183 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:11.668231 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260808 17:48:12.436065 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:12.435834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:43367)
0808 17:48:12.435984 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:12.436018 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260808 17:48:12.513939 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:12.513738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:40653)
0808 17:48:12.513856 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:12.513894 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260808 17:48:12.668968 21757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:12.668740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:34853)
0808 17:48:12.668881 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:12.668923 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260808 17:48:13.260397 21758 raft_consensus.cc:482] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:13.260529 21758 raft_consensus.cc:504] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:13.260720 21758 leader_election.cc:288] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129), 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
W20260808 17:48:13.261193 21171 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
W20260808 17:48:13.261271 21169 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
I20260808 17:48:13.261386 21169 leader_election.cc:302] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fbba6faf9e24bacb7f6f0b8b19cd2e1; no voters: 30cfdc793f7d409ca81ce400d68c168d, 8e03a3df12d643e49ae99ed29c97043e
I20260808 17:48:13.261483 21758 raft_consensus.cc:2726] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:48:13.436759 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:13.436526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:43141)
0808 17:48:13.436672 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:13.436709 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260808 17:48:13.514632 21761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:13.514435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:60821)
0808 17:48:13.514551 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:13.514589 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260808 17:48:13.669653 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:13.669435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:60935)
0808 17:48:13.669574 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:13.669610 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260808 17:48:14.437554 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:14.437312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:54883)
0808 17:48:14.437464 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:14.437504 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
I20260808 17:48:14.515386 21766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:14.515163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:55961)
0808 17:48:14.515307 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:14.515339 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260808 17:48:14.670460 21769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:14.670145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:59305)
0808 17:48:14.670316 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:14.670361 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
I20260808 17:48:15.438413 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:15.438096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:34295)
0808 17:48:15.438292 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:15.438335 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":116,"threads_started":1}
I20260808 17:48:15.516041 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:15.515828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:38263)
0808 17:48:15.515951 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:15.515999 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260808 17:48:15.671212 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:15.670937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:59623)
0808 17:48:15.671093 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:15.671168 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260808 17:48:15.671202 21774 raft_consensus.cc:482] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:15.671311 21774 raft_consensus.cc:504] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:15.671525 21774 leader_election.cc:288] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129), 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
W20260808 17:48:15.671890 21169 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
W20260808 17:48:15.671964 21171 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
I20260808 17:48:15.672042 21171 leader_election.cc:302] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fbba6faf9e24bacb7f6f0b8b19cd2e1; no voters: 30cfdc793f7d409ca81ce400d68c168d, 8e03a3df12d643e49ae99ed29c97043e
I20260808 17:48:15.672132 21774 raft_consensus.cc:2726] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:48:16.439102 21775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:16.438847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:47185)
0808 17:48:16.439012 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:16.439053 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260808 17:48:16.516800 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:16.516577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:38777)
0808 17:48:16.516706 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:16.516750 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260808 17:48:16.672070 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:16.671838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:39811)
0808 17:48:16.671978 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:16.672021 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260808 17:48:17.439751 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:17.439543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:46073)
0808 17:48:17.439665 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:17.439702 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260808 17:48:17.517454 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:17.517236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:47231)
0808 17:48:17.517373 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:17.517408 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260808 17:48:17.672833 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:17.672618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:41285)
0808 17:48:17.672744 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:17.672787 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260808 17:48:18.440366 21782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:18.440161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:38719)
0808 17:48:18.440285 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:18.440316 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260808 17:48:18.518117 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:18.517919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:34517)
0808 17:48:18.518039 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:18.518075 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260808 17:48:18.673502 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:18.673310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:36529)
0808 17:48:18.673423 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:18.673458 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260808 17:48:19.256305 21786 raft_consensus.cc:482] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:19.256408 21786 raft_consensus.cc:504] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:19.256704 21786 leader_election.cc:288] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129), 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
W20260808 17:48:19.257004 21171 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
W20260808 17:48:19.257076 21169 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
I20260808 17:48:19.257164 21169 leader_election.cc:302] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fbba6faf9e24bacb7f6f0b8b19cd2e1; no voters: 30cfdc793f7d409ca81ce400d68c168d, 8e03a3df12d643e49ae99ed29c97043e
I20260808 17:48:19.257246 21786 raft_consensus.cc:2726] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:48:19.441231 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:19.440990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:44827)
0808 17:48:19.441135 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:19.441180 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260808 17:48:19.518844 21792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:19.518572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:36495)
0808 17:48:19.518711 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:19.518761 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260808 17:48:19.674350 21793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:19.673996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:55967)
0808 17:48:19.674241 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:19.674287 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":81,"threads_started":1}
I20260808 17:48:20.441929 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:20.441725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:49799)
0808 17:48:20.441853 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:20.441882 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260808 17:48:20.519585 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:20.519327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:52789)
0808 17:48:20.519464 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:20.519538 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260808 17:48:20.675107 21796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:20.674874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:53945)
0808 17:48:20.675026 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:20.675061 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260808 17:48:21.442720 21797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:21.442508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:34225)
0808 17:48:21.442635 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:21.442672 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260808 17:48:21.520255 21799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:21.520045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:60745)
0808 17:48:21.520172 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:21.520209 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260808 17:48:21.675822 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:21.675601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:40021)
0808 17:48:21.675731 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:21.675770 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260808 17:48:22.443499 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:22.443233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:52689)
0808 17:48:22.443407 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:22.443451 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260808 17:48:22.508503 21802 raft_consensus.cc:482] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:22.508602 21802 raft_consensus.cc:504] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e03a3df12d643e49ae99ed29c97043e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45129 } } peers { permanent_uuid: "30cfdc793f7d409ca81ce400d68c168d" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 45017 } } peers { permanent_uuid: "6fbba6faf9e24bacb7f6f0b8b19cd2e1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 44639 } }
I20260808 17:48:22.508771 21802 leader_election.cc:288] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129), 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017)
W20260808 17:48:22.509193 21171 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e03a3df12d643e49ae99ed29c97043e (127.29.62.194:45129): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
W20260808 17:48:22.509302 21169 leader_election.cc:341] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30cfdc793f7d409ca81ce400d68c168d (127.29.62.195:45017): Not found: Tablet not found: 892cc958836744c0ad281efca839b2f3
I20260808 17:48:22.509337 21169 leader_election.cc:302] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fbba6faf9e24bacb7f6f0b8b19cd2e1; no voters: 30cfdc793f7d409ca81ce400d68c168d, 8e03a3df12d643e49ae99ed29c97043e
I20260808 17:48:22.509433 21802 raft_consensus.cc:2726] T 892cc958836744c0ad281efca839b2f3 P 6fbba6faf9e24bacb7f6f0b8b19cd2e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:48:22.520925 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:22.520696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:55961)
0808 17:48:22.520835 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:22.520874 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260808 17:48:22.676591 21804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:22.676352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:43449)
0808 17:48:22.676505 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:22.676544 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260808 17:48:23.444221 21805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:23.444003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:52203)
0808 17:48:23.444131 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:23.444172 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260808 17:48:23.521606 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:23.521378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:51491)
0808 17:48:23.521523 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:23.521561 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260808 17:48:23.677302 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:23.677088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:37125)
0808 17:48:23.677216 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:23.677253 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260808 17:48:24.444943 21808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:24.444715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:59311)
0808 17:48:24.444852 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:24.444893 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260808 17:48:24.522457 21809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:24.522107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:41467)
0808 17:48:24.522283 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:24.522329 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260808 17:48:24.678164 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:24.677911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:40077)
0808 17:48:24.678078 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:24.678117 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260808 17:48:25.445706 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:25.445489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.193:39299)
0808 17:48:25.445622 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:25.445659 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260808 17:48:25.523185 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:25.522978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.194:43245)
0808 17:48:25.523107 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:25.523139 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260808 17:48:25.675555 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 21101
I20260808 17:48:25.685566 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:25.684641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40337 (local address 127.29.62.195:47813)
0808 17:48:25.685447 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:25.685503 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":738,"threads_started":1}
I20260808 17:48:25.690995 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 21340
I20260808 17:48:25.702108 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 21546
2026-08-08T17:48:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18556 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:48:25 dist-test-slave-tk9z krb5kdc[21818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:48:25 dist-test-slave-tk9z krb5kdc[21818](info): set up 2 sockets
Aug 08 17:48:25 dist-test-slave-tk9z krb5kdc[21818](info): commencing operation
krb5kdc: starting...
W20260808 17:48:26.955724 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:48:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:48:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:48:27.077204 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40321
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:27.160804 21840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:27.161038 21840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:27.161077 21840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:27.162211 21840 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 17:48:27.162276 21840 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 17:48:27.162297 21840 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 17:48:27.162317 21840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:27.162350 21840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:27.162393 21840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:27.162426 21840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:27.162453 21840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:27.163954 21840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:27.164244 21840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:27.164476 21840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:27.169638 21845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:27.169620 21851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:27.169620 21847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:27.169648 21840 server_base.cc:1034] running on GCE node
I20260808 17:48:27.170038 21840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:27.170339 21840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:27.171551 21840 hybrid_clock.cc:648] HybridClock initialized: now 1786211307171533 us; error 31 us; skew 500 ppm
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:27.177341 21840 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:48:27.177838 21840 webserver.cc:460] Webserver started at http://127.29.62.254:36505/ using document root <none> and password file <none>
I20260808 17:48:27.178076 21840 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:27.178120 21840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:27.178215 21840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:27.179368 21840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "33b83c040dd749a4bdd2883a24cbd840"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "f7581aa155d8f4d648b946696de9f4a3"
server_key_iv: "c7b756d46bd729ea571bc57708249191"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.179689 21840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "33b83c040dd749a4bdd2883a24cbd840"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "f7581aa155d8f4d648b946696de9f4a3"
server_key_iv: "c7b756d46bd729ea571bc57708249191"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.180989 21840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:48:27.181708 21863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.181883 21840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:27.181977 21840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "33b83c040dd749a4bdd2883a24cbd840"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "f7581aa155d8f4d648b946696de9f4a3"
server_key_iv: "c7b756d46bd729ea571bc57708249191"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.182035 21840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:27.199728 21840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:27.201550 21840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:27.201660 21840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:27.205642 21840 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40321
I20260808 17:48:27.205703 21942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40321 every 8 connection(s)
I20260808 17:48:27.206058 21840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:48:27.207034 21943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:27.209378 21943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Bootstrap starting.
I20260808 17:48:27.209996 21943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:27.210307 21943 log.cc:824] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:27.211038 21943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: No bootstrap required, opened a new log
I20260808 17:48:27.212255 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 21840
I20260808 17:48:27.212340 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:48:27.212551 29947 external_mini_cluster.cc:1428] Setting key dd72308b7ff2defc62936c4347c3de89
I20260808 17:48:27.212788 21943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:48:27.212950 21943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:27.212989 21943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b83c040dd749a4bdd2883a24cbd840, State: Initialized, Role: FOLLOWER
I20260808 17:48:27.213096 21943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:48:27.213184 21943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:48:27.213218 21943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:48:27.213265 21943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:27.214102 21943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:48:27.214231 21943 leader_election.cc:302] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33b83c040dd749a4bdd2883a24cbd840; no voters:
I20260808 17:48:27.214435 21943 leader_election.cc:288] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:48:27.214473 21949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:27.214591 21949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 1 LEADER]: Becoming Leader. State: Replica: 33b83c040dd749a4bdd2883a24cbd840, State: Running, Role: LEADER
I20260808 17:48:27.214725 21943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:48:27.214731 21949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:48:27.215155 21950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33b83c040dd749a4bdd2883a24cbd840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } } }
I20260808 17:48:27.215379 21950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:27.215322 21951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33b83c040dd749a4bdd2883a24cbd840. Latest consensus state: current_term: 1 leader_uuid: "33b83c040dd749a4bdd2883a24cbd840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } } }
I20260808 17:48:27.215534 21951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:27.215778 21961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:48:27.216338 21961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:48:27.217907 21961 catalog_manager.cc:1347] Generated new cluster ID: a4bdfca611a64dc5bdd577e58a51fa48
I20260808 17:48:27.217988 21961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:27.224277 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.213067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.29.62.254:40321)
0808 17:48:27.213278 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:27.213287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:27.213324 (+ 37us) server_negotiation.cc:407] Connection header received
0808 17:48:27.213448 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:27.213453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:27.213500 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:27.213633 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:27.214285 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.215768 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.216908 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.217291 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.217408 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.220291 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:27.220295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:27.220296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:27.220308 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:27.222199 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.222657 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.222659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.222660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.223115 (+ 455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.223352 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.223353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.223353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.223638 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:27.223780 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:27.223917 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 17:48:27.224056 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260808 17:48:27.233722 21961 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:48:27.234508 21961 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:48:27.240288 21961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Generated new TSK 0
I20260808 17:48:27.240466 21961 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:48:27.273983 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40321
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:27.358068 21976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:27.358261 21976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:27.358284 21976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:27.359580 21976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:27.359624 21976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:27.359651 21976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:48:27.361040 21976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:27.361234 21976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:27.361411 21976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:27.366340 21985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:27.366494 21989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:27.366361 21983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:27.366541 21976 server_base.cc:1034] running on GCE node
I20260808 17:48:27.366814 21976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:27.367055 21976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:27.368228 21976 hybrid_clock.cc:648] HybridClock initialized: now 1786211307368215 us; error 29 us; skew 500 ppm
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:27.371203 21976 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:48:27.371577 21976 webserver.cc:460] Webserver started at http://127.29.62.193:40179/ using document root <none> and password file <none>
I20260808 17:48:27.371837 21976 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:27.371883 21976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:27.372033 21976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:27.373112 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "ddfb5b722d154662b946b3b884ad3722"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a2d45a3f12a8bafd2325982a74a47b43"
server_key_iv: "e6896849127d277cff92631e6c3cf865"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.373510 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "ddfb5b722d154662b946b3b884ad3722"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a2d45a3f12a8bafd2325982a74a47b43"
server_key_iv: "e6896849127d277cff92631e6c3cf865"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.374923 21976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:48:27.375672 22000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.375842 21976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.375953 21976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "ddfb5b722d154662b946b3b884ad3722"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a2d45a3f12a8bafd2325982a74a47b43"
server_key_iv: "e6896849127d277cff92631e6c3cf865"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.376032 21976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:27.387087 21976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:27.389394 21976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:27.389565 21976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:27.389791 21976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:27.390097 21976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:27.390152 21976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.390194 21976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:27.390223 21976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.396061 21976 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:38253
I20260808 17:48:27.396139 22167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:38253 every 8 connection(s)
I20260808 17:48:27.396469 21976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:48:27.399622 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 21976
I20260808 17:48:27.399712 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:48:27.399919 29947 external_mini_cluster.cc:1428] Setting key 88fe7015388290d7090fb2005e8e5169
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:27.406454 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.397310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:59857 (local address 127.29.62.254:40321)
0808 17:48:27.397391 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:27.397394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:27.397585 (+ 191us) server_negotiation.cc:407] Connection header received
0808 17:48:27.398141 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:27.398143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:27.398160 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:27.398212 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:27.399254 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.399881 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.400902 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.401127 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.401215 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.403512 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:27.403513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:27.403514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:27.403524 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:27.404928 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.405377 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.405378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.405379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.405450 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.405711 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.405712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.405712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.405842 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:27.405915 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:27.406219 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 17:48:27.406316 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:48:27.406673 22172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.397260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:59857)
0808 17:48:27.397453 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:27.397498 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:27.398032 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:27.398295 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:27.398301 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:27.398952 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:27.399147 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.399152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.399975 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.399977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.400802 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.400805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.401229 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.401230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.401364 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.401887 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:27.401894 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:27.403403 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:27.405046 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:27.405054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:27.405056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:27.405292 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:27.405521 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:27.405522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:27.405522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:27.405639 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:27.405918 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:27.405926 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:27.406074 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 17:48:27.406182 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 17:48:27.406348 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260808 17:48:27.407207 22168 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40321
I20260808 17:48:27.407308 22168 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:27.407524 22168 heartbeater.cc:511] Master 127.29.62.254:40321 requested a full tablet report, sending...
I20260808 17:48:27.408130 21885 ts_manager.cc:194] Registered new tserver with Master: ddfb5b722d154662b946b3b884ad3722 (127.29.62.193:38253)
I20260808 17:48:27.409147 21885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:59857
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:48:27.427050 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40321
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:27.510310 22176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:27.510536 22176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:27.510560 22176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:27.511735 22176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:27.511783 22176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:27.511809 22176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:48:27.513275 22176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:27.513541 22176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:27.513780 22176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:27.518914 22181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:27.518919 22185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:27.518934 22182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:27.519150 22176 server_base.cc:1034] running on GCE node
I20260808 17:48:27.519345 22176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:27.519582 22176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:27.520743 22176 hybrid_clock.cc:648] HybridClock initialized: now 1786211307520724 us; error 30 us; skew 500 ppm
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:27.523258 22176 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:48:27.523725 22176 webserver.cc:460] Webserver started at http://127.29.62.194:33875/ using document root <none> and password file <none>
I20260808 17:48:27.523968 22176 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:27.524047 22176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:27.524201 22176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:27.525218 22176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "759a443ba15e42b5b6f963a44bc4ae14"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "e84809a0ec2c7296e0af083aaec134b2"
server_key_iv: "a2ef1de59fe99caecc231d641e5d335e"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.525585 22176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "759a443ba15e42b5b6f963a44bc4ae14"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "e84809a0ec2c7296e0af083aaec134b2"
server_key_iv: "a2ef1de59fe99caecc231d641e5d335e"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.526980 22176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:48:27.527700 22194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.527869 22176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:48:27.527984 22176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "759a443ba15e42b5b6f963a44bc4ae14"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "e84809a0ec2c7296e0af083aaec134b2"
server_key_iv: "a2ef1de59fe99caecc231d641e5d335e"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.528059 22176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:27.542208 22176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:27.544260 22176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:27.544425 22176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:27.544654 22176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:27.544968 22176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:27.545033 22176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.545075 22176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:27.545102 22176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.550936 22176 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:40589
I20260808 17:48:27.550988 22352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:40589 every 8 connection(s)
I20260808 17:48:27.551302 22176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:48:27.552249 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 22176
I20260808 17:48:27.552372 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:48:27.552524 29947 external_mini_cluster.cc:1428] Setting key c262238ac60658bcca85221084eb1e98
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:27.562018 22355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.551983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:41687)
0808 17:48:27.552173 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:27.552215 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:27.552750 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:27.553017 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:27.553022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:27.553654 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:27.553846 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.553851 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.554852 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.554853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.556118 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.556122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.556486 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.556487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.556631 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.557131 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:27.557141 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:27.558844 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:27.560710 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:27.560716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:27.560718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:27.560932 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:27.561158 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:27.561158 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:27.561158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:27.561252 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:27.561539 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:27.561544 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:27.561660 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 17:48:27.561736 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:48:27.561868 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":118,"threads_started":1}
I20260808 17:48:27.562012 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.551983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:41687 (local address 127.29.62.254:40321)
0808 17:48:27.552057 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:27.552059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:27.552314 (+ 255us) server_negotiation.cc:407] Connection header received
0808 17:48:27.552856 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:27.552857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:27.552868 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:27.552939 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:27.553955 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.554763 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.556210 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.556396 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.556459 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.558947 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:27.558949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:27.558949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:27.558958 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:27.560625 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.561013 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.561014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.561015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.561086 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.561319 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.561320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.561320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.561470 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:27.561546 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:27.561803 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 17:48:27.561918 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:48:27.562546 22353 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40321
I20260808 17:48:27.562664 22353 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:27.562871 22353 heartbeater.cc:511] Master 127.29.62.254:40321 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
I20260808 17:48:27.563364 21885 ts_manager.cc:194] Registered new tserver with Master: 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
I20260808 17:48:27.563735 21885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:41687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:48:27.579449 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40321
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:27.662479 22358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:27.662727 22358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:27.662765 22358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:27.663911 22358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:27.663977 22358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:27.664021 22358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:48:27.665479 22358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:27.665748 22358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:27.665982 22358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:27.671011 22368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:27.671036 22371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:27.671137 22358 server_base.cc:1034] running on GCE node
W20260808 17:48:27.671018 22369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:27.671525 22358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:27.671800 22358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:27.672959 22358 hybrid_clock.cc:648] HybridClock initialized: now 1786211307672956 us; error 36 us; skew 500 ppm
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:27.675530 22358 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:48:27.675992 22358 webserver.cc:460] Webserver started at http://127.29.62.195:33631/ using document root <none> and password file <none>
I20260808 17:48:27.676218 22358 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:27.676283 22358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:27.676440 22358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:48:27.677512 22358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "d896d2d1e46c41929798c81e5ce96d30"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a4eca87d9e24be990acff4752d76a911"
server_key_iv: "73d2a884a353066674708d20d99b7a53"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.677884 22358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "d896d2d1e46c41929798c81e5ce96d30"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a4eca87d9e24be990acff4752d76a911"
server_key_iv: "73d2a884a353066674708d20d99b7a53"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.679391 22358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:48:27.680115 22383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.680292 22358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.680397 22358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "d896d2d1e46c41929798c81e5ce96d30"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a4eca87d9e24be990acff4752d76a911"
server_key_iv: "73d2a884a353066674708d20d99b7a53"
server_key_version: "encryptionkey@0"
I20260808 17:48:27.680503 22358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:27.698205 22358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:27.700119 22358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:27.700282 22358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:27.700525 22358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:27.700826 22358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:27.700882 22358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.700919 22358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:27.700964 22358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:27.707231 22358 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:40877
I20260808 17:48:27.707296 22556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:40877 every 8 connection(s)
I20260808 17:48:27.707600 22358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:48:27 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:27.714918 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 22358
I20260808 17:48:27.715063 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:48:27.715261 29947 external_mini_cluster.cc:1428] Setting key 8ec68257b40e94b320e5de5f075c833b
I20260808 17:48:27.716950 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.708332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:33087 (local address 127.29.62.254:40321)
0808 17:48:27.708402 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:27.708403 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:27.708588 (+ 185us) server_negotiation.cc:407] Connection header received
0808 17:48:27.709057 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:27.709058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:27.709068 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:27.709124 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:27.710009 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.710521 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.711520 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.711700 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.711749 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.713818 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:27.713819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:27.713820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:27.713827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:27.715405 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.715956 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.715957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.715958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.716027 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.716368 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.716368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.716369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.716476 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:27.716552 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:27.716808 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 17:48:27.716889 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:48:27.717121 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.708288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:33087)
0808 17:48:27.708461 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:27.708502 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:27.708977 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:27.709175 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:27.709178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:27.709742 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:27.709935 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.709940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.710595 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.710595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.711446 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.711449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.711769 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.711770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.711867 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.712275 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:27.712283 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:27.713732 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:27.715519 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:27.715525 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:27.715527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:27.715889 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:27.716147 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:27.716148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:27.716149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:27.716297 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:27.716551 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:27.716556 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:27.716662 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 17:48:27.716742 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 17:48:27.716923 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260808 17:48:27.717540 22558 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40321
I20260808 17:48:27.717617 22558 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:27.717746 22558 heartbeater.cc:511] Master 127.29.62.254:40321 requested a full tablet report, sending...
I20260808 17:48:27.718040 21885 ts_manager.cc:194] Registered new tserver with Master: d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877)
I20260808 17:48:27.718461 21885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:33087
I20260808 17:48:27.726987 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:48:27.734068 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.728297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.29.62.254:40321)
0808 17:48:27.728357 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:27.728359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:27.728489 (+ 130us) server_negotiation.cc:407] Connection header received
0808 17:48:27.728575 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:27.728576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:27.728586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:27.728644 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:27.729174 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.729649 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.730395 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.730631 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.730711 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.731870 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:27.731871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:27.731871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:27.731877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:27.733191 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.733484 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.733484 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.733485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.733533 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:27.733664 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:27.733665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:27.733665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:27.733794 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:27.733865 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:27.733955 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 17:48:27.734008 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:48:27.735666 21885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:48:27.737180 21885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:48:27.744014 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.739551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.29.62.195:40877)
0808 17:48:27.739888 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:27.739892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:27.739912 (+ 20us) server_negotiation.cc:407] Connection header received
0808 17:48:27.739982 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:27.739984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:27.740000 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:27.740086 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:27.740383 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.741599 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.742848 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.743582 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.743664 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.743725 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:27.743884 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 17:48:27.743927 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":100,"threads_started":1}
I20260808 17:48:27.744045 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.739372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:38253 (local address 127.0.0.1:57194)
0808 17:48:27.739955 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:27.739963 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:27.740039 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:27.740210 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:27.740211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:27.740212 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:27.740328 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.740329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.741647 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.741648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.742912 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.742915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.743754 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.743754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.743804 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.743807 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:48:27.743857 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:48:27.743933 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":83,"thread_start_us":66,"threads_started":1}
I20260808 17:48:27.744045 22574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.739372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:40877 (local address 127.0.0.1:34112)
0808 17:48:27.739746 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:27.739777 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:27.739868 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:27.740195 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:27.740198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:27.740201 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:27.740289 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.740291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.741681 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.741682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.742746 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.742749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.743667 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.743668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.743765 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.743781 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:48:27.743857 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:48:27.743933 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":34,"thread_start_us":65,"threads_started":1}
I20260808 17:48:27.744314 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.739515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.29.62.193:38253)
0808 17:48:27.739739 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:27.739745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:27.740046 (+ 301us) server_negotiation.cc:407] Connection header received
0808 17:48:27.740103 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:27.740105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:27.740121 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:27.740205 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:27.740414 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.741532 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.743004 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.743678 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.743747 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.743793 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:27.744175 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 17:48:27.744225 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":105,"threads_started":1}
I20260808 17:48:27.744932 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.739582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:40589 (local address 127.0.0.1:44142)
0808 17:48:27.740394 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:27.740404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:27.740450 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:27.740638 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:27.740638 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:27.740641 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:27.740678 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.740680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.741515 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.741516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.742912 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:27.742915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.744700 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.744701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:27.744764 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.744769 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:48:27.744819 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:48:27.744865 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":61,"thread_start_us":41,"threads_started":1}
I20260808 17:48:27.745250 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:27.739703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44142 (local address 127.29.62.194:40589)
0808 17:48:27.739892 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:27.739898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:27.740451 (+ 553us) server_negotiation.cc:407] Connection header received
0808 17:48:27.740532 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:27.740534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:27.740549 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:27.740642 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:27.740739 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.741414 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.743856 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:27.744632 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:27.744699 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:27.744752 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:27.745112 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 17:48:27.745164 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260808 17:48:27.745182 22078 tablet_service.cc:1467] Processing CreateTablet for tablet 33a62697e6e4419689e1ff249533ff77 (DEFAULT_TABLE table=test-table [id=259c79a478f2411eafc0f0e9705d0acf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:27.745244 22453 tablet_service.cc:1467] Processing CreateTablet for tablet 33a62697e6e4419689e1ff249533ff77 (DEFAULT_TABLE table=test-table [id=259c79a478f2411eafc0f0e9705d0acf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:27.745582 22078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a62697e6e4419689e1ff249533ff77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:27.745617 22453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a62697e6e4419689e1ff249533ff77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:27.745849 22262 tablet_service.cc:1467] Processing CreateTablet for tablet 33a62697e6e4419689e1ff249533ff77 (DEFAULT_TABLE table=test-table [id=259c79a478f2411eafc0f0e9705d0acf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:27.746152 22262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33a62697e6e4419689e1ff249533ff77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:27.748287 22584 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Bootstrap starting.
I20260808 17:48:27.748981 22587 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Bootstrap starting.
I20260808 17:48:27.749027 22584 tablet_bootstrap.cc:654] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:27.749384 22589 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Bootstrap starting.
I20260808 17:48:27.749720 22587 tablet_bootstrap.cc:654] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:27.749826 22584 log.cc:824] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:27.750036 22589 tablet_bootstrap.cc:654] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:27.750272 22587 log.cc:824] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:27.750692 22589 log.cc:824] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:27.751039 22584 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: No bootstrap required, opened a new log
I20260808 17:48:27.751113 22587 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: No bootstrap required, opened a new log
I20260808 17:48:27.751117 22584 ts_tablet_manager.cc:1397] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:48:27.751179 22587 ts_tablet_manager.cc:1397] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:48:27.751608 22589 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: No bootstrap required, opened a new log
I20260808 17:48:27.751690 22589 ts_tablet_manager.cc:1397] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:48:27.753014 22589 raft_consensus.cc:348] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:27.753185 22589 raft_consensus.cc:374] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:27.753080 22584 raft_consensus.cc:348] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:27.753080 22587 raft_consensus.cc:348] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:27.753230 22589 raft_consensus.cc:729] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 759a443ba15e42b5b6f963a44bc4ae14, State: Initialized, Role: FOLLOWER
I20260808 17:48:27.753252 22587 raft_consensus.cc:374] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:27.753252 22584 raft_consensus.cc:374] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:27.753288 22584 raft_consensus.cc:729] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfb5b722d154662b946b3b884ad3722, State: Initialized, Role: FOLLOWER
I20260808 17:48:27.753288 22587 raft_consensus.cc:729] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d896d2d1e46c41929798c81e5ce96d30, State: Initialized, Role: FOLLOWER
I20260808 17:48:27.753366 22589 consensus_queue.cc:260] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:27.753432 22584 consensus_queue.cc:260] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:27.753427 22587 consensus_queue.cc:260] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:27.753654 22589 ts_tablet_manager.cc:1428] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:48:27.753703 22587 ts_tablet_manager.cc:1428] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:48:27.753703 22584 ts_tablet_manager.cc:1428] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:48:27.753790 22353 heartbeater.cc:503] Master 127.29.62.254:40321 was elected leader, sending a full tablet report...
I20260808 17:48:27.753819 22558 heartbeater.cc:503] Master 127.29.62.254:40321 was elected leader, sending a full tablet report...
I20260808 17:48:27.754081 22168 heartbeater.cc:503] Master 127.29.62.254:40321 was elected leader, sending a full tablet report...
W20260808 17:48:27.801841 22354 tablet.cc:2369] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:48:27.897260 22170 tablet.cc:2369] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:48:27.958171 22560 tablet.cc:2369] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:48:28.102277 22600 raft_consensus.cc:482] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:28.102439 22600 raft_consensus.cc:504] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:28.102767 22600 leader_election.cc:288] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddfb5b722d154662b946b3b884ad3722 (127.29.62.193:38253), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
I20260808 17:48:28.105731 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:28.103091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:38399 (local address 127.29.62.193:38253)
0808 17:48:28.103163 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:28.103165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:28.103175 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:48:28.103206 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:28.103208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:28.103219 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:28.103282 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:28.103473 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.104013 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.104967 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.105345 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.105415 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:28.105423 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:28.105614 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 17:48:28.105659 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:48:28.105752 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:28.103179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:40593 (local address 127.29.62.194:40589)
0808 17:48:28.103231 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:28.103233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:28.103297 (+ 64us) server_negotiation.cc:407] Connection header received
0808 17:48:28.103369 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:28.103370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:28.103380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:28.103448 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:28.103615 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.104100 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.105034 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.105393 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.105466 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:28.105475 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:28.105649 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 17:48:28.105690 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:48:28.105728 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:28.102957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:38253 (local address 127.29.62.195:38399)
0808 17:48:28.103048 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:28.103063 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:28.103127 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:28.103306 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:28.103307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:28.103310 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:28.103390 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:28.103392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.104097 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.104098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:28.104871 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:28.104873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.105429 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.105431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:28.105529 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:28.105533 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:28.105583 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:48:28.105633 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 17:48:28.105748 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:28.103088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:40589 (local address 127.29.62.195:40593)
0808 17:48:28.103247 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:28.103257 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:28.103304 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:28.103473 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:28.103474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:28.103476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:28.103546 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:28.103547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.104183 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.104184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:28.104950 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:28.104953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.105488 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.105489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:28.105553 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:28.105557 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:28.105634 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 17:48:28.105667 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260808 17:48:28.106105 22103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a62697e6e4419689e1ff249533ff77" candidate_uuid: "d896d2d1e46c41929798c81e5ce96d30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfb5b722d154662b946b3b884ad3722" is_pre_election: true
I20260808 17:48:28.106118 22284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a62697e6e4419689e1ff249533ff77" candidate_uuid: "d896d2d1e46c41929798c81e5ce96d30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "759a443ba15e42b5b6f963a44bc4ae14" is_pre_election: true
I20260808 17:48:28.106292 22284 raft_consensus.cc:2454] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d896d2d1e46c41929798c81e5ce96d30 in term 0.
I20260808 17:48:28.106297 22103 raft_consensus.cc:2454] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d896d2d1e46c41929798c81e5ce96d30 in term 0.
I20260808 17:48:28.106578 22384 leader_election.cc:302] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 759a443ba15e42b5b6f963a44bc4ae14, d896d2d1e46c41929798c81e5ce96d30; no voters:
I20260808 17:48:28.106715 22600 raft_consensus.cc:2781] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:48:28.106792 22600 raft_consensus.cc:482] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:48:28.106828 22600 raft_consensus.cc:3037] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:28.107569 22600 raft_consensus.cc:504] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:28.107719 22600 leader_election.cc:288] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [CANDIDATE]: Term 1 election: Requested vote from peers ddfb5b722d154662b946b3b884ad3722 (127.29.62.193:38253), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
I20260808 17:48:28.107878 22103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a62697e6e4419689e1ff249533ff77" candidate_uuid: "d896d2d1e46c41929798c81e5ce96d30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddfb5b722d154662b946b3b884ad3722"
I20260808 17:48:28.107937 22284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33a62697e6e4419689e1ff249533ff77" candidate_uuid: "d896d2d1e46c41929798c81e5ce96d30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "759a443ba15e42b5b6f963a44bc4ae14"
I20260808 17:48:28.107988 22103 raft_consensus.cc:3037] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:28.108022 22284 raft_consensus.cc:3037] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:28.108731 22284 raft_consensus.cc:2454] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d896d2d1e46c41929798c81e5ce96d30 in term 1.
I20260808 17:48:28.108731 22103 raft_consensus.cc:2454] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d896d2d1e46c41929798c81e5ce96d30 in term 1.
I20260808 17:48:28.108892 22384 leader_election.cc:302] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 759a443ba15e42b5b6f963a44bc4ae14, d896d2d1e46c41929798c81e5ce96d30; no voters:
I20260808 17:48:28.108975 22600 raft_consensus.cc:2781] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:28.109115 22600 raft_consensus.cc:686] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 1 LEADER]: Becoming Leader. State: Replica: d896d2d1e46c41929798c81e5ce96d30, State: Running, Role: LEADER
I20260808 17:48:28.109242 22600 consensus_queue.cc:237] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:28.110114 21884 catalog_manager.cc:5495] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 reported cstate change: term changed from 0 to 1, leader changed from <none> to d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195). New cstate: current_term: 1 leader_uuid: "d896d2d1e46c41929798c81e5ce96d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } health_report { overall_health: HEALTHY } } }
I20260808 17:48:28.187011 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:28.184374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34122 (local address 127.29.62.195:40877)
0808 17:48:28.184433 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:28.184435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:28.184445 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:48:28.184555 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:28.184556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:28.184579 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:28.184654 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:48:28.184814 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.185405 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.186157 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:28.186414 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:28.186478 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:28.186651 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:48:28.186768 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:48:28.186871 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:28.186890 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 17:48:28.186912 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:48:28.189169 22103 raft_consensus.cc:1261] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Refusing update from remote peer d896d2d1e46c41929798c81e5ce96d30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:28.189169 22284 raft_consensus.cc:1261] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 1 FOLLOWER]: Refusing update from remote peer d896d2d1e46c41929798c81e5ce96d30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:28.189541 22600 consensus_queue.cc:1035] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:28.189707 22602 consensus_queue.cc:1035] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:28.192550 22608 mvcc.cc:205] Tried to move back new op lower bound from 7316321518338699264 to 7316321518016069632. Current Snapshot: MvccSnapshot[applied={T|T < 7316321518338699264}]
I20260808 17:48:28.193553 22607 mvcc.cc:205] Tried to move back new op lower bound from 7316321518338699264 to 7316321518016069632. Current Snapshot: MvccSnapshot[applied={T|T < 7316321518338699264 or (T in {7316321518338699264})}]
I20260808 17:48:28.194442 21883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:28.194566 21883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:28.195369 21883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Sending DeleteTablet for 3 replicas of tablet 33a62697e6e4419689e1ff249533ff77
I20260808 17:48:28.195729 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 21976
I20260808 17:48:28.195731 22453 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a62697e6e4419689e1ff249533ff77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:28 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:34112
I20260808 17:48:28.195742 22078 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a62697e6e4419689e1ff249533ff77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:28 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:57194
I20260808 17:48:28.195804 22262 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a62697e6e4419689e1ff249533ff77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:28 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:44142
I20260808 17:48:28.198175 22620 tablet_replica.cc:330] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: stopping tablet replica
I20260808 17:48:28.198266 22620 raft_consensus.cc:2229] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:28.198362 22620 raft_consensus.cc:2258] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:28.201767 22620 ts_tablet_manager.cc:1905] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:28.203545 22620 ts_tablet_manager.cc:1918] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:28.203611 22620 log.cc:1196] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/33a62697e6e4419689e1ff249533ff77
I20260808 17:48:28.203830 22620 ts_tablet_manager.cc:1939] T 33a62697e6e4419689e1ff249533ff77 P 759a443ba15e42b5b6f963a44bc4ae14: Deleting consensus metadata
I20260808 17:48:28.204252 21864 catalog_manager.cc:4841] TS 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589): tablet 33a62697e6e4419689e1ff249533ff77 (table test-table [id=259c79a478f2411eafc0f0e9705d0acf]) successfully deleted
I20260808 17:48:28.208230 22619 tablet_replica.cc:330] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: stopping tablet replica
I20260808 17:48:28.208357 22619 raft_consensus.cc:2229] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:48:28.208479 22619 raft_consensus.cc:2258] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:28.208909 22619 ts_tablet_manager.cc:1905] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:28.210141 22619 ts_tablet_manager.cc:1918] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:28.210237 22619 log.cc:1196] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/33a62697e6e4419689e1ff249533ff77
I20260808 17:48:28.210590 22619 ts_tablet_manager.cc:1939] T 33a62697e6e4419689e1ff249533ff77 P d896d2d1e46c41929798c81e5ce96d30: Deleting consensus metadata
I20260808 17:48:28.211022 21867 catalog_manager.cc:4841] TS d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877): tablet 33a62697e6e4419689e1ff249533ff77 (table test-table [id=259c79a478f2411eafc0f0e9705d0acf]) successfully deleted
W20260808 17:48:28.214973 21864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:38253 (error 108)
I20260808 17:48:28.215566 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:28.215404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:38253 (local address 127.0.0.1:57210)
0808 17:48:28.215459 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:28.215529 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:38253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:48:28.215639 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 22176
W20260808 17:48:28.222589 21864 catalog_manager.cc:4584] TS ddfb5b722d154662b946b3b884ad3722 (127.29.62.193:38253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33a62697e6e4419689e1ff249533ff77: Network error: Client connection negotiation failed: client connection to 127.29.62.193:38253: connect: Connection refused (error 111)
I20260808 17:48:28.232079 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 22358
I20260808 17:48:28.244463 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 21840
I20260808 17:48:48.254160 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40321
--webserver_interface=127.29.62.254
--webserver_port=36505
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:48:48.338961 22623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:48.339167 22623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:48.339190 22623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:48.340353 22623 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 17:48:48.340394 22623 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 17:48:48.340408 22623 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 17:48:48.340420 22623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:48:48.340457 22623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:48.340490 22623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:48.340508 22623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:48:48.340531 22623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:48:48.342065 22623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=36505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:48.342329 22623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:48.342618 22623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:48.351895 22634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:48.351934 22623 server_base.cc:1034] running on GCE node
W20260808 17:48:48.351933 22637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:48.352025 22635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:48.352348 22623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:48.352674 22623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:48.353821 22623 hybrid_clock.cc:648] HybridClock initialized: now 1786211328353812 us; error 33 us; skew 500 ppm
Aug 08 17:48:48 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211328, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:48.360203 22623 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:48:48.360725 22623 webserver.cc:460] Webserver started at http://127.29.62.254:36505/ using document root <none> and password file <none>
I20260808 17:48:48.360957 22623 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:48.361016 22623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:48.362484 22623 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:48:48.363148 22647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.363336 22623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:48:48.363472 22623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "33b83c040dd749a4bdd2883a24cbd840"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "f7581aa155d8f4d648b946696de9f4a3"
server_key_iv: "c7b756d46bd729ea571bc57708249191"
server_key_version: "encryptionkey@0"
I20260808 17:48:48.363931 22623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:48.383860 22623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:48.385901 22623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:48.386080 22623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:48.390503 22623 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40321
I20260808 17:48:48.390545 22723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40321 every 8 connection(s)
I20260808 17:48:48.390996 22623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:48:48.392607 22725 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:48:48.393324 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Bootstrap starting.
I20260808 17:48:48.401001 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 22623
I20260808 17:48:48.401324 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:38253
--local_ip_for_outbound_sockets=127.29.62.193
--tserver_master_addrs=127.29.62.254:40321
--webserver_port=40179
--webserver_interface=127.29.62.193
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:48.402565 22725 log.cc:824] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:48.405110 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:48:48.405448 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Bootstrap complete.
I20260808 17:48:48.407882 22725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:48:48.408119 22725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b83c040dd749a4bdd2883a24cbd840, State: Initialized, Role: FOLLOWER
I20260808 17:48:48.408300 22725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:48:48.408367 22725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:48:48.408440 22725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:48:48.408504 22725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:48:48.409572 22725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:48:48.409667 22725 leader_election.cc:302] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33b83c040dd749a4bdd2883a24cbd840; no voters:
I20260808 17:48:48.409893 22725 leader_election.cc:288] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:48:48.409934 22731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:48:48.410120 22731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 2 LEADER]: Becoming Leader. State: Replica: 33b83c040dd749a4bdd2883a24cbd840, State: Running, Role: LEADER
I20260808 17:48:48.410187 22725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:48:48.410331 22731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:48:48.410699 22731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33b83c040dd749a4bdd2883a24cbd840. Latest consensus state: current_term: 2 leader_uuid: "33b83c040dd749a4bdd2883a24cbd840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } } }
I20260808 17:48:48.410784 22734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "33b83c040dd749a4bdd2883a24cbd840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } } }
I20260808 17:48:48.410878 22731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:48.410889 22734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: This master's current role is: LEADER
I20260808 17:48:48.411125 22743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:48:48.412089 22743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:48:48.412249 22743 catalog_manager.cc:1259] Loaded cluster ID: a4bdfca611a64dc5bdd577e58a51fa48
I20260808 17:48:48.412537 22743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:48:48.413688 22743 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:48:48.413856 22743 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Loaded TSK: 0
I20260808 17:48:48.422775 22743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Generated new TSK 1
I20260808 17:48:48.422868 22743 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:48:48.486797 22730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:48.487027 22730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:48.487063 22730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:48.488193 22730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:48.488263 22730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:48.488307 22730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:48:48.489755 22730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:38253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=40179
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:48.490023 22730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:48.490265 22730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:48.495241 22764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:48.495283 22762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:48.495348 22730 server_base.cc:1034] running on GCE node
W20260808 17:48:48.495512 22766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:48.495793 22730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:48.496124 22730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:48.497282 22730 hybrid_clock.cc:648] HybridClock initialized: now 1786211328497264 us; error 28 us; skew 500 ppm
Aug 08 17:48:48 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211328, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:48.499845 22730 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:48:48.500329 22730 webserver.cc:460] Webserver started at http://127.29.62.193:40179/ using document root <none> and password file <none>
I20260808 17:48:48.500562 22730 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:48.500648 22730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:48.502069 22730 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:48.502883 22773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.503069 22730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:48:48.503176 22730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "ddfb5b722d154662b946b3b884ad3722"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a2d45a3f12a8bafd2325982a74a47b43"
server_key_iv: "e6896849127d277cff92631e6c3cf865"
server_key_version: "encryptionkey@0"
I20260808 17:48:48.503561 22730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:48.513089 22730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:48.515377 22730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:48.515542 22730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:48.515776 22730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:48.516295 22784 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:48:48.517778 22730 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:48:48.517829 22730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:48:48.517864 22730 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:48:48.518637 22730 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:48:48.518684 22730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 17:48:48.518728 22784 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Bootstrap starting.
I20260808 17:48:48.526614 22730 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:38253
I20260808 17:48:48.527184 22730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:48:48.527859 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 22730
I20260808 17:48:48.528231 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:40589
--local_ip_for_outbound_sockets=127.29.62.194
--tserver_master_addrs=127.29.62.254:40321
--webserver_port=33875
--webserver_interface=127.29.62.194
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:48:48.530948 22784 log.cc:824] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:48.531280 22949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:38253 every 8 connection(s)
I20260808 17:48:48.533951 22784 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:48:48.534341 22784 tablet_bootstrap.cc:492] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Bootstrap complete.
I20260808 17:48:48.535739 22784 ts_tablet_manager.cc:1397] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.012s
I20260808 17:48:48.537549 22784 raft_consensus.cc:348] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:48.538022 22784 raft_consensus.cc:729] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfb5b722d154662b946b3b884ad3722, State: Initialized, Role: FOLLOWER
I20260808 17:48:48.538267 22784 consensus_queue.cc:260] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } }
I20260808 17:48:48.539034 22784 ts_tablet_manager.cc:1428] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
Aug 08 17:48:48 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211328, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:48.545251 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.530669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:33199)
0808 17:48:48.531558 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:48.531599 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:48.532265 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:48.532933 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:48.532937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:48.533336 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:48.533640 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.533646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.535528 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.535529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.536892 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.536895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.537883 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.537884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.538032 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.538710 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:48.538721 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:48.540698 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:48.543375 (+ 2677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:48.543383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:48.543386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:48.543692 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:48.544005 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:48.544006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:48.544006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:48.544148 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:48.544609 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:48.544615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:48.544765 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 17:48:48.544877 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 17:48:48.545050 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"thread_start_us":81,"threads_started":1}
I20260808 17:48:48.545325 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.530786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:33199 (local address 127.29.62.254:40321)
0808 17:48:48.531000 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.531011 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.531724 (+ 713us) server_negotiation.cc:407] Connection header received
0808 17:48:48.532527 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.532535 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.532605 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.532776 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:48.533806 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.535386 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.537002 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.537660 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.537741 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.540845 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:48.540849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:48.540850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:48.540863 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:48.543284 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.543798 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.543801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.543801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.543924 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.544226 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.544227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.544228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.544532 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:48.544627 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:48.544961 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 17:48:48.545111 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":95,"threads_started":1}
I20260808 17:48:48.545985 22951 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40321
I20260808 17:48:48.546106 22951 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:48.546322 22951 heartbeater.cc:511] Master 127.29.62.254:40321 requested a full tablet report, sending...
I20260808 17:48:48.547202 22675 ts_manager.cc:194] Registered new tserver with Master: ddfb5b722d154662b946b3b884ad3722 (127.29.62.193:38253)
I20260808 17:48:48.548491 22675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:33199
I20260808 17:48:48.549492 22951 heartbeater.cc:503] Master 127.29.62.254:40321 was elected leader, sending a full tablet report...
I20260808 17:48:48.554425 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.548139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.29.62.193:38253)
0808 17:48:48.550462 (+ 2323us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.550466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.550474 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:48:48.550504 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.550506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.550519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.550592 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:48.550806 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.551850 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.553321 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.554013 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.554090 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.554140 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:48.554305 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 17:48:48.554342 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2293,"thread_start_us":75,"threads_started":1}
I20260808 17:48:48.554445 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.547886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:38253 (local address 127.0.0.1:33160)
0808 17:48:48.550197 (+ 2311us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:48.550234 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:48.550314 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:48.550624 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:48.550628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:48.550631 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:48.550703 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.550705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.551942 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.551943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.553228 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.553231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.554104 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.554105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.554187 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.554201 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 17:48:48.554275 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:48:48.554327 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2284,"thread_start_us":99,"threads_started":1}
I20260808 17:48:48.555032 22854 tablet_service.cc:1514] Processing DeleteTablet for tablet 33a62697e6e4419689e1ff249533ff77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:28 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:33160
I20260808 17:48:48.555631 22961 tablet_replica.cc:330] stopping tablet replica
I20260808 17:48:48.555739 22961 raft_consensus.cc:2229] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:48.555807 22961 raft_consensus.cc:2258] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:48.556114 22961 ts_tablet_manager.cc:1905] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:48.558274 22961 ts_tablet_manager.cc:1918] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:48.558349 22961 log.cc:1196] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal/wals/33a62697e6e4419689e1ff249533ff77
I20260808 17:48:48.558621 22961 ts_tablet_manager.cc:1939] T 33a62697e6e4419689e1ff249533ff77 P ddfb5b722d154662b946b3b884ad3722: Deleting consensus metadata
I20260808 17:48:48.559142 22649 catalog_manager.cc:4841] TS ddfb5b722d154662b946b3b884ad3722 (127.29.62.193:38253): tablet 33a62697e6e4419689e1ff249533ff77 (table test-table [id=259c79a478f2411eafc0f0e9705d0acf]) successfully deleted
W20260808 17:48:48.619560 22953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:48.619752 22953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:48.619776 22953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:48.620820 22953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:48.620853 22953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:48.620878 22953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:48:48.622224 22953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:40589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=33875
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:48.622457 22953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:48.622641 22953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:48:48.627568 22953 server_base.cc:1034] running on GCE node
W20260808 17:48:48.627527 22971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:48.627465 22967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:48.627477 22968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:48.627842 22953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:48.628057 22953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:48.629222 22953 hybrid_clock.cc:648] HybridClock initialized: now 1786211328629204 us; error 33 us; skew 500 ppm
Aug 08 17:48:48 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211328, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:48.631779 22953 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:48:48.632228 22953 webserver.cc:460] Webserver started at http://127.29.62.194:33875/ using document root <none> and password file <none>
I20260808 17:48:48.632431 22953 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:48.632484 22953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:48.633728 22953 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:48:48.634284 22982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.634486 22953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:48:48.634590 22953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "759a443ba15e42b5b6f963a44bc4ae14"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "e84809a0ec2c7296e0af083aaec134b2"
server_key_iv: "a2ef1de59fe99caecc231d641e5d335e"
server_key_version: "encryptionkey@0"
I20260808 17:48:48.634931 22953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:48.645083 22953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:48.646975 22953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:48.647132 22953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:48.647356 22953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:48.647670 22953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:48.647702 22953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.647724 22953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:48.647773 22953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.653543 22953 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:40589
I20260808 17:48:48.653630 23132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:40589 every 8 connection(s)
I20260808 17:48:48.653954 22953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:48:48.656098 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 22953
I20260808 17:48:48.656366 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:40877
--local_ip_for_outbound_sockets=127.29.62.195
--tserver_master_addrs=127.29.62.254:40321
--webserver_port=33631
--webserver_interface=127.29.62.195
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:48:48 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211328, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:48.664237 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.654708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:42369 (local address 127.29.62.254:40321)
0808 17:48:48.654762 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.654765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.654950 (+ 185us) server_negotiation.cc:407] Connection header received
0808 17:48:48.655499 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.655501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.655519 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.655577 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:48.656737 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.657243 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.658437 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.658653 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.658715 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.661150 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:48.661151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:48.661152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:48.661160 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:48.662693 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.663164 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.663165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.663165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.663230 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.663511 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.663513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.663513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.663638 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:48.663713 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:48.664029 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 17:48:48.664136 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:48:48.664405 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.654667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:42369)
0808 17:48:48.654815 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:48.654863 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:48.655382 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:48.655644 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:48.655648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:48.656322 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:48.656632 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.656638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.657351 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.657353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.658311 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.658314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.658744 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.658745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.658898 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.659444 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:48.659455 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:48.661044 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:48.662815 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:48.662824 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:48.662827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:48.663067 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:48.663301 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:48.663302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:48.663303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:48.663421 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:48.663718 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:48.663726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:48.663870 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 17:48:48.663983 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 17:48:48.664156 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260808 17:48:48.664906 23133 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40321
I20260808 17:48:48.665030 23133 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:48.665262 23133 heartbeater.cc:511] Master 127.29.62.254:40321 requested a full tablet report, sending...
I20260808 17:48:48.665704 22675 ts_manager.cc:194] Registered new tserver with Master: 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
I20260808 17:48:48.666109 22675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:42369
W20260808 17:48:48.749933 23136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:48:48.750152 23136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:48:48.750177 23136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:48:48.751490 23136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:48:48.751539 23136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:48:48.751569 23136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:48:48.753187 23136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:40877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=33631
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:48:48.753455 23136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:48:48.753710 23136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:48:48.759374 23144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:48.759414 23136 server_base.cc:1034] running on GCE node
W20260808 17:48:48.759383 23141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:48:48.759383 23142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:48:48.759797 23136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:48:48.760067 23136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:48:48.761243 23136 hybrid_clock.cc:648] HybridClock initialized: now 1786211328761226 us; error 27 us; skew 500 ppm
Aug 08 17:48:48 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211328, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:48:48.763978 23136 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:48:48.764506 23136 webserver.cc:460] Webserver started at http://127.29.62.195:33631/ using document root <none> and password file <none>
I20260808 17:48:48.764757 23136 fs_manager.cc:359] Metadata directory not provided
I20260808 17:48:48.764849 23136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:48:48.766320 23136 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:48:48.767030 23153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.767210 23136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:48:48.767340 23136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "d896d2d1e46c41929798c81e5ce96d30"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a4eca87d9e24be990acff4752d76a911"
server_key_iv: "73d2a884a353066674708d20d99b7a53"
server_key_version: "encryptionkey@0"
I20260808 17:48:48.767755 23136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:48:48.793119 23136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:48:48.795331 23136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:48:48.795531 23136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:48:48.795840 23136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:48:48.796182 23136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:48:48.796238 23136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.796274 23136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:48:48.796321 23136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.802259 23136 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:40877
I20260808 17:48:48.802301 23326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:40877 every 8 connection(s)
I20260808 17:48:48.802708 23136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:48:48.803122 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 23136
Aug 08 17:48:48 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211328, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:48:48.811316 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.803568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.29.62.254:40321)
0808 17:48:48.803652 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.803655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.803779 (+ 124us) server_negotiation.cc:407] Connection header received
0808 17:48:48.803928 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.803929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.803941 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.804001 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:48.804680 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.805523 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.806682 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.806897 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.806955 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.808390 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:48.808392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:48.808392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:48.808399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:48.809970 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.810424 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.810425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.810425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.810495 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.810724 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.810724 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.810724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.810819 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:48.811059 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:48.811104 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 17:48:48.811183 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 17:48:48.813920 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.803642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:57579 (local address 127.29.62.254:40321)
0808 17:48:48.803747 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.803749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.803869 (+ 120us) server_negotiation.cc:407] Connection header received
0808 17:48:48.804512 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.804514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.804528 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.804609 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:48.805843 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.806646 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.808061 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.808249 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.808305 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.810870 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:48.810871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:48.810871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:48.810877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:48.812518 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.812932 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.812933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.812934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.813008 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.813271 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.813272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.813272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.813384 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:48.813454 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:48.813740 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 17:48:48.813829 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":59,"threads_started":1}
I20260808 17:48:48.814160 23331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.803551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:57579)
0808 17:48:48.803735 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:48.803777 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:48.804396 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:48.804724 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:48.804728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:48.805510 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:48:48.805758 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.805764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.806740 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.806741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.807951 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.807955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.808341 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.808342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.808487 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.809099 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:48:48.809106 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:48:48.810784 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:48:48.812610 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:48.812616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:48.812618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:48.812840 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:48.813075 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:48:48.813076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:48:48.813077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:48:48.813194 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:48:48.813459 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:48:48.813466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:48:48.813607 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 17:48:48.813717 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 17:48:48.813951 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260808 17:48:48.814620 23327 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40321
I20260808 17:48:48.814725 23327 heartbeater.cc:464] Registering TS with master...
I20260808 17:48:48.814908 23327 heartbeater.cc:511] Master 127.29.62.254:40321 requested a full tablet report, sending...
I20260808 17:48:48.815299 22675 ts_manager.cc:194] Registered new tserver with Master: d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877)
I20260808 17:48:48.815667 22675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:57579
I20260808 17:48:48.822815 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:48:48.825433 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.823163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.29.62.254:40321)
0808 17:48:48.823218 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.823220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.823345 (+ 125us) server_negotiation.cc:407] Connection header received
0808 17:48:48.823396 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.823396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.823428 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.823501 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:48:48.823617 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.824161 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.824867 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.825051 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.825098 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.825235 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:48:48.825274 (+ 39us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 17:48:48.825390 (+ 116us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39420: token expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 17:48:48.825510 23332 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39420: token expired
W20260808 17:48:48.825594 22569 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.62.254:40321)
I20260808 17:48:48.831493 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.825827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.29.62.254:40321)
0808 17:48:48.825862 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.825863 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.825871 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:48:48.825924 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.825925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.825935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.826005 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:48:48.826541 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.827058 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.827751 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.828014 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.828095 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.829188 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:48:48.829190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:48:48.829190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:48:48.829197 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:48:48.830313 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.830707 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.830708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.830708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.830769 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:48:48.830970 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:48:48.830971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:48:48.830972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:48:48.831103 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:48:48.831210 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:48.831316 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 17:48:48.831397 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:48:48.832671 22675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:48:48.834262 22675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:48:48.836431 22854 tablet_service.cc:1467] Processing CreateTablet for tablet 42dbe43fdc0e406db696d7416032c107 (DEFAULT_TABLE table=test-table [id=e081aa01ab5940339f522077bfa545c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:48.836649 22854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dbe43fdc0e406db696d7416032c107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:48.838196 22784 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Bootstrap starting.
I20260808 17:48:48.838599 22784 tablet_bootstrap.cc:654] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:48.839476 22784 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: No bootstrap required, opened a new log
I20260808 17:48:48.839529 22784 ts_tablet_manager.cc:1397] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 17:48:48.839628 22784 raft_consensus.cc:348] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:48.839694 22784 raft_consensus.cc:374] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:48.839740 22784 raft_consensus.cc:729] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfb5b722d154662b946b3b884ad3722, State: Initialized, Role: FOLLOWER
I20260808 17:48:48.839784 22784 consensus_queue.cc:260] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:48.839898 22784 ts_tablet_manager.cc:1428] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:48:48.840122 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.836098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:40589 (local address 127.0.0.1:36518)
0808 17:48:48.836269 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:48.836281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:48.836320 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:48.836854 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:48.836855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:48.836857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:48.836899 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.836900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.838012 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.838013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.838981 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.838983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.839879 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.839879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.839936 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.839940 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:48.839969 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 17:48:48.840053 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":42}
I20260808 17:48:48.840514 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.836445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36518 (local address 127.29.62.194:40589)
0808 17:48:48.836655 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.836660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.836671 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:48:48.836735 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.836737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.836755 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.836858 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:48.836957 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.837925 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.839064 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.839776 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.840283 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.840356 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:48.840382 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 17:48:48.840421 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":108,"threads_started":1}
I20260808 17:48:48.841246 23050 tablet_service.cc:1467] Processing CreateTablet for tablet 42dbe43fdc0e406db696d7416032c107 (DEFAULT_TABLE table=test-table [id=e081aa01ab5940339f522077bfa545c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:48.841591 23050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dbe43fdc0e406db696d7416032c107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:48.841995 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.836196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.29.62.195:40877)
0808 17:48:48.838442 (+ 2246us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:48.838446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:48.838457 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:48:48.838498 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:48.838500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:48.838517 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:48.838606 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:48.838746 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.839753 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.841032 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.841643 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.841727 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.841773 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:48.841903 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 17:48:48.841933 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2205,"thread_start_us":101,"threads_started":1}
I20260808 17:48:48.842008 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:48.836098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:40877 (local address 127.0.0.1:50842)
0808 17:48:48.836315 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:48.836326 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:48.836388 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:48.838622 (+ 2234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:48.838623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:48.838626 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:48.838662 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.838663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.839870 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.839872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.840944 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:48.840947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:48.841749 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:48.841750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:48.841812 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:48.841816 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:48.841875 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:48:48.841911 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":15,"thread_start_us":118,"threads_started":1}
I20260808 17:48:48.844358 23339 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Bootstrap starting.
I20260808 17:48:48.845117 23339 tablet_bootstrap.cc:654] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:48.845095 23219 tablet_service.cc:1467] Processing CreateTablet for tablet 42dbe43fdc0e406db696d7416032c107 (DEFAULT_TABLE table=test-table [id=e081aa01ab5940339f522077bfa545c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:48:48.845458 23219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dbe43fdc0e406db696d7416032c107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:48:48.845729 23339 log.cc:824] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:48.846570 23339 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: No bootstrap required, opened a new log
I20260808 17:48:48.846647 23339 ts_tablet_manager.cc:1397] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:48:48.848728 23339 raft_consensus.cc:348] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:48.848910 23339 raft_consensus.cc:374] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:48.848969 23339 raft_consensus.cc:729] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 759a443ba15e42b5b6f963a44bc4ae14, State: Initialized, Role: FOLLOWER
I20260808 17:48:48.849092 23339 consensus_queue.cc:260] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:48.849385 23339 ts_tablet_manager.cc:1428] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:48:48.849411 23133 heartbeater.cc:503] Master 127.29.62.254:40321 was elected leader, sending a full tablet report...
I20260808 17:48:48.849587 23341 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Bootstrap starting.
I20260808 17:48:48.850221 23341 tablet_bootstrap.cc:654] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Neither blocks nor log segments found. Creating new log.
I20260808 17:48:48.850780 23341 log.cc:824] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Log is configured to *not* fsync() on all Append() calls
I20260808 17:48:48.851500 23341 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: No bootstrap required, opened a new log
I20260808 17:48:48.851588 23341 ts_tablet_manager.cc:1397] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:48:48.852869 23341 raft_consensus.cc:348] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:48.853029 23341 raft_consensus.cc:374] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:48:48.853075 23341 raft_consensus.cc:729] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d896d2d1e46c41929798c81e5ce96d30, State: Initialized, Role: FOLLOWER
I20260808 17:48:48.853191 23341 consensus_queue.cc:260] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:48.853462 23341 ts_tablet_manager.cc:1428] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:48:48.853533 23327 heartbeater.cc:503] Master 127.29.62.254:40321 was elected leader, sending a full tablet report...
W20260808 17:48:48.904520 23134 tablet.cc:2369] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:48:49.022657 22957 raft_consensus.cc:482] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:48:49.022800 22957 raft_consensus.cc:504] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:49.023248 22957 leader_election.cc:288] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
I20260808 17:48:49.026228 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:49.023405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:40877 (local address 127.29.62.193:55921)
0808 17:48:49.023452 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:49.023466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:49.023537 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:49.023717 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:49.023718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:49.023720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:49.023795 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:49.023797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.024645 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.024646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:49.025402 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:49.025404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.025968 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.025970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:49.026042 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:49.026046 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:49.026111 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:48:49.026156 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 17:48:49.026228 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:49.023529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:55921 (local address 127.29.62.195:40877)
0808 17:48:49.023576 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:49.023579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:49.023588 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:48:49.023620 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:49.023621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:49.023633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:49.023695 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:49.023876 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.024560 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.025501 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.025872 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.025959 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:49.025969 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:49.026117 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 17:48:49.026156 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:48:49.026527 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:49.023539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:35261 (local address 127.29.62.194:40589)
0808 17:48:49.023583 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:49.023585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:49.023698 (+ 113us) server_negotiation.cc:407] Connection header received
0808 17:48:49.023745 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:49.023746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:49.023756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:49.023811 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:48:49.024336 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.024792 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.025714 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.026093 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.026183 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:49.026192 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:49.026375 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 17:48:49.026461 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:48:49.026669 23259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbe43fdc0e406db696d7416032c107" candidate_uuid: "ddfb5b722d154662b946b3b884ad3722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d896d2d1e46c41929798c81e5ce96d30" is_pre_election: true
I20260808 17:48:49.026765 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:49.023430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:40589 (local address 127.29.62.193:35261)
0808 17:48:49.023600 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:49.023611 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:48:49.023685 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:48:49.023825 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:48:49.023826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:48:49.023844 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:48:49.024235 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:49.024238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.024890 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.024891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:49.025620 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:48:49.025622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.026199 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.026200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:48:49.026260 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:49.026264 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:48:49.026643 (+ 379us) client_negotiation.cc:240] Negotiation successful
0808 17:48:49.026690 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":24,"thread_start_us":84,"threads_started":1}
I20260808 17:48:49.026856 23259 raft_consensus.cc:2454] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddfb5b722d154662b946b3b884ad3722 in term 0.
I20260808 17:48:49.027143 22779 leader_election.cc:302] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d896d2d1e46c41929798c81e5ce96d30, ddfb5b722d154662b946b3b884ad3722; no voters:
I20260808 17:48:49.027159 23074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbe43fdc0e406db696d7416032c107" candidate_uuid: "ddfb5b722d154662b946b3b884ad3722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "759a443ba15e42b5b6f963a44bc4ae14" is_pre_election: true
I20260808 17:48:49.027334 23074 raft_consensus.cc:2454] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddfb5b722d154662b946b3b884ad3722 in term 0.
I20260808 17:48:49.027310 22957 raft_consensus.cc:2781] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:48:49.027381 22957 raft_consensus.cc:482] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:48:49.027418 22957 raft_consensus.cc:3037] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:49.028199 22957 raft_consensus.cc:504] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:49.028362 22957 leader_election.cc:288] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 1 election: Requested vote from peers d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
I20260808 17:48:49.028528 23074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbe43fdc0e406db696d7416032c107" candidate_uuid: "ddfb5b722d154662b946b3b884ad3722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "759a443ba15e42b5b6f963a44bc4ae14"
I20260808 17:48:49.028594 23074 raft_consensus.cc:3037] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:49.029357 23074 raft_consensus.cc:2454] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddfb5b722d154662b946b3b884ad3722 in term 1.
I20260808 17:48:49.029489 23259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbe43fdc0e406db696d7416032c107" candidate_uuid: "ddfb5b722d154662b946b3b884ad3722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d896d2d1e46c41929798c81e5ce96d30"
I20260808 17:48:49.029548 22777 leader_election.cc:302] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 759a443ba15e42b5b6f963a44bc4ae14, ddfb5b722d154662b946b3b884ad3722; no voters:
I20260808 17:48:49.029582 23259 raft_consensus.cc:3037] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:48:49.029667 22957 raft_consensus.cc:2781] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:48:49.029855 22957 raft_consensus.cc:686] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 LEADER]: Becoming Leader. State: Replica: ddfb5b722d154662b946b3b884ad3722, State: Running, Role: LEADER
I20260808 17:48:49.029942 22957 consensus_queue.cc:237] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:48:49.030614 23259 raft_consensus.cc:2454] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddfb5b722d154662b946b3b884ad3722 in term 1.
I20260808 17:48:49.030822 22675 catalog_manager.cc:5495] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 reported cstate change: term changed from 0 to 1, leader changed from <none> to ddfb5b722d154662b946b3b884ad3722 (127.29.62.193). New cstate: current_term: 1 leader_uuid: "ddfb5b722d154662b946b3b884ad3722" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } health_report { overall_health: HEALTHY } } }
W20260808 17:48:49.031862 22952 tablet.cc:2369] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:48:49.053357 23329 tablet.cc:2369] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:48:49.068888 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:49.066349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33164 (local address 127.29.62.193:38253)
0808 17:48:49.066550 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 17:48:49.066552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:48:49.066559 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:48:49.066598 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:48:49.066599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:48:49.066619 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:48:49.066711 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:48:49.066896 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.067411 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.068215 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:48:49.068408 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:48:49.068463 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:48:49.068611 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:48:49.068715 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:48:49.068769 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:48:49.068804 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 17:48:49.068823 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":126,"threads_started":1}
I20260808 17:48:49.070572 23259 raft_consensus.cc:1261] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 1 FOLLOWER]: Refusing update from remote peer ddfb5b722d154662b946b3b884ad3722: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:49.070593 23074 raft_consensus.cc:1261] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 1 FOLLOWER]: Refusing update from remote peer ddfb5b722d154662b946b3b884ad3722: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:48:49.070935 23349 consensus_queue.cc:1035] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:49.071115 22957 consensus_queue.cc:1035] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [LEADER]: Connected to new peer: Peer: permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:48:49.076512 22672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:49.076638 22672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:48:49.077440 22672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Sending DeleteTablet for 3 replicas of tablet 42dbe43fdc0e406db696d7416032c107
I20260808 17:48:49.077776 22854 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dbe43fdc0e406db696d7416032c107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:49 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:33160
I20260808 17:48:49.077838 23050 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dbe43fdc0e406db696d7416032c107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:49 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:36518
I20260808 17:48:49.078038 23219 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dbe43fdc0e406db696d7416032c107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:48:49 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:50842
I20260808 17:48:49.078096 23372 tablet_replica.cc:330] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: stopping tablet replica
I20260808 17:48:49.078181 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 22730
I20260808 17:48:49.078284 23372 raft_consensus.cc:2229] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:49.078418 23372 raft_consensus.cc:2258] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:49.078899 23372 ts_tablet_manager.cc:1905] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:49.081171 23372 ts_tablet_manager.cc:1918] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:49.081260 23372 log.cc:1196] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/42dbe43fdc0e406db696d7416032c107
I20260808 17:48:49.081612 23372 ts_tablet_manager.cc:1939] T 42dbe43fdc0e406db696d7416032c107 P 759a443ba15e42b5b6f963a44bc4ae14: Deleting consensus metadata
I20260808 17:48:49.084743 22649 catalog_manager.cc:4841] TS 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589): tablet 42dbe43fdc0e406db696d7416032c107 (table test-table [id=e081aa01ab5940339f522077bfa545c0]) successfully deleted
I20260808 17:48:49.085592 23373 tablet_replica.cc:330] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: stopping tablet replica
I20260808 17:48:49.085724 23373 raft_consensus.cc:2229] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:48:49.085843 23373 raft_consensus.cc:2258] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:48:49.086457 23373 ts_tablet_manager.cc:1905] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:48:49.088052 23373 ts_tablet_manager.cc:1918] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:48:49.088117 23373 log.cc:1196] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/42dbe43fdc0e406db696d7416032c107
I20260808 17:48:49.088394 23373 ts_tablet_manager.cc:1939] T 42dbe43fdc0e406db696d7416032c107 P d896d2d1e46c41929798c81e5ce96d30: Deleting consensus metadata
I20260808 17:48:49.088862 22652 catalog_manager.cc:4841] TS d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877): tablet 42dbe43fdc0e406db696d7416032c107 (table test-table [id=e081aa01ab5940339f522077bfa545c0]) successfully deleted
W20260808 17:48:49.092068 22649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:38253 (error 108)
I20260808 17:48:49.092520 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 22953
I20260808 17:48:49.092705 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:48:49.092550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:38253 (local address 127.0.0.1:33170)
0808 17:48:49.092614 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 17:48:49.092662 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:38253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 17:48:49.097452 22649 catalog_manager.cc:4584] TS ddfb5b722d154662b946b3b884ad3722 (127.29.62.193:38253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42dbe43fdc0e406db696d7416032c107: Network error: Client connection negotiation failed: client connection to 127.29.62.193:38253: connect: Connection refused (error 111)
I20260808 17:48:49.102610 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 23136
I20260808 17:48:49.113168 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 22623
I20260808 17:49:09.124444 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40321
--webserver_interface=127.29.62.254
--webserver_port=36505
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:09.209587 23377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:09.209781 23377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:09.209803 23377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:09.210902 23377 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 17:49:09.210937 23377 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 17:49:09.210949 23377 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 17:49:09.210960 23377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:09.210974 23377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:09.210986 23377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:09.210999 23377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:09.211014 23377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:09.212450 23377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=36505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:09.212646 23377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:09.212812 23377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:09.217758 23388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:09.217767 23386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:09.217767 23385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:09.218004 23377 server_base.cc:1034] running on GCE node
I20260808 17:49:09.218267 23377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:09.218603 23377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:09.219751 23377 hybrid_clock.cc:648] HybridClock initialized: now 1786211349219739 us; error 35 us; skew 500 ppm
Aug 08 17:49:09 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211349, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:09.226306 23377 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:09.226806 23377 webserver.cc:460] Webserver started at http://127.29.62.254:36505/ using document root <none> and password file <none>
I20260808 17:49:09.227057 23377 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:09.227128 23377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:09.228615 23377 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:09.229317 23397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.229487 23377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:09.229598 23377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "33b83c040dd749a4bdd2883a24cbd840"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "f7581aa155d8f4d648b946696de9f4a3"
server_key_iv: "c7b756d46bd729ea571bc57708249191"
server_key_version: "encryptionkey@0"
I20260808 17:49:09.229975 23377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:09.243525 23377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:09.245986 23377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:09.246167 23377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:09.250510 23377 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40321
I20260808 17:49:09.250571 23469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40321 every 8 connection(s)
I20260808 17:49:09.251000 23377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:09.252552 23471 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:49:09.253170 23471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Bootstrap starting.
I20260808 17:49:09.260975 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 23377
I20260808 17:49:09.261284 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:38253
--local_ip_for_outbound_sockets=127.29.62.193
--tserver_master_addrs=127.29.62.254:40321
--webserver_port=40179
--webserver_interface=127.29.62.193
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:49:09.262012 23471 log.cc:824] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:09.265046 23471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:49:09.265445 23471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Bootstrap complete.
I20260808 17:49:09.269624 23471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:49:09.270036 23471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b83c040dd749a4bdd2883a24cbd840, State: Initialized, Role: FOLLOWER
I20260808 17:49:09.270229 23471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:49:09.270357 23471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:09.270443 23471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:09.270507 23471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 2 FOLLOWER]: Advancing to term 3
I20260808 17:49:09.272164 23471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:49:09.272302 23471 leader_election.cc:302] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33b83c040dd749a4bdd2883a24cbd840; no voters:
I20260808 17:49:09.272533 23471 leader_election.cc:288] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 17:49:09.272662 23475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 17:49:09.272841 23475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [term 3 LEADER]: Becoming Leader. State: Replica: 33b83c040dd749a4bdd2883a24cbd840, State: Running, Role: LEADER
I20260808 17:49:09.272864 23471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:09.272943 23475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } }
I20260808 17:49:09.273319 23476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "33b83c040dd749a4bdd2883a24cbd840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } } }
I20260808 17:49:09.273397 23476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:09.273663 23477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33b83c040dd749a4bdd2883a24cbd840. Latest consensus state: current_term: 3 leader_uuid: "33b83c040dd749a4bdd2883a24cbd840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b83c040dd749a4bdd2883a24cbd840" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40321 } } }
I20260808 17:49:09.273797 23477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:09.273849 23483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:09.274787 23483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:09.274894 23483 catalog_manager.cc:1259] Loaded cluster ID: a4bdfca611a64dc5bdd577e58a51fa48
I20260808 17:49:09.274926 23483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:09.276142 23483 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:09.276332 23483 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Loaded TSK: 0
I20260808 17:49:09.276386 23483 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 33b83c040dd749a4bdd2883a24cbd840: Loaded TSK: 1
F20260808 17:49:09.276651 23483 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb8a55169ad google::LogMessage::Fail() at ??:0
@ 0x7fb8a551988a google::LogMessage::SendToLog() at ??:0
@ 0x7fb8a55164de google::LogMessage::Flush() at ??:0
@ 0x7fb8a5517a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb8a7c83104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb8a7c80d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb8ad4741cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb8ad47448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb8ad474594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fb8ad43ac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb8ad444ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb8a73e40d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb8a73d9954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb8a823d6db start_thread at ??:0
@ 0x7fb8a401461f clone at ??:0
W20260808 17:49:09.365280 23474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:09.365510 23474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:09.365550 23474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:09.366775 23474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:09.366840 23474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:09.366899 23474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:49:09.368474 23474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:38253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=40179
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:09.368743 23474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:09.368990 23474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:09.374195 23527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:09.374212 23529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:09.374212 23531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:09.374560 23474 server_base.cc:1034] running on GCE node
I20260808 17:49:09.374773 23474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:09.375093 23474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:09.376303 23474 hybrid_clock.cc:648] HybridClock initialized: now 1786211349376279 us; error 38 us; skew 500 ppm
Aug 08 17:49:09 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211349, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:09.379482 23474 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:49:09.380182 23474 webserver.cc:460] Webserver started at http://127.29.62.193:40179/ using document root <none> and password file <none>
I20260808 17:49:09.380445 23474 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:09.380522 23474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:09.382077 23474 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:09.382962 23539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.383155 23474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:49:09.383273 23474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "ddfb5b722d154662b946b3b884ad3722"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a2d45a3f12a8bafd2325982a74a47b43"
server_key_iv: "e6896849127d277cff92631e6c3cf865"
server_key_version: "encryptionkey@0"
I20260808 17:49:09.383682 23474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:09.408443 23474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:09.411199 23474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:09.411371 23474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:09.411641 23474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:09.412163 23552 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:49:09.413702 23474 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:49:09.413766 23474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:49:09.413810 23474 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:49:09.414877 23474 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:49:09.414927 23474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:09.414984 23552 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Bootstrap starting.
I20260808 17:49:09.422474 23474 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:38253
I20260808 17:49:09.422552 23699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:38253 every 8 connection(s)
I20260808 17:49:09.422960 23474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:49:09.425627 23552 log.cc:824] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:09.428028 23552 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:49:09.428309 23552 tablet_bootstrap.cc:492] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Bootstrap complete.
I20260808 17:49:09.429036 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 23474
I20260808 17:49:09.429319 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:40589
--local_ip_for_outbound_sockets=127.29.62.194
--tserver_master_addrs=127.29.62.254:40321
--webserver_port=33875
--webserver_interface=127.29.62.194
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:49:09.429481 23552 ts_tablet_manager.cc:1397] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260808 17:49:09.431357 23552 raft_consensus.cc:348] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:49:09.431632 23552 raft_consensus.cc:729] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddfb5b722d154662b946b3b884ad3722, State: Initialized, Role: FOLLOWER
I20260808 17:49:09.431797 23552 consensus_queue.cc:260] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:49:09.432101 23552 ts_tablet_manager.cc:1428] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
W20260808 17:49:09.489164 23704 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:49:09.423696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:44683)
0808 17:49:09.423925 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.424003 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:09.424744 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:09.488998 (+ 64254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
W20260808 17:49:09.489434 23700 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:40321 (0 consecutive failures): Network error: Failed to ping master at 127.29.62.254:40321: Client connection negotiation failed: client connection to 127.29.62.254:40321: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:49:09.489794 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:09.489666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:55163)
0808 17:49:09.489724 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.489772 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 17:49:09.490175 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:09.490081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:42623)
0808 17:49:09.490126 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.490150 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 17:49:09.490295 23700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:49:09.519649 23707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:09.519886 23707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:09.519929 23707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:09.521044 23707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:09.521114 23707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:09.521159 23707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:49:09.522717 23707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:40589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=33875
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:09.522982 23707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:09.523216 23707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:09.528155 23714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:09.528167 23719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:09.528167 23715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:09.528545 23707 server_base.cc:1034] running on GCE node
I20260808 17:49:09.528761 23707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:09.528960 23707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:09.530097 23707 hybrid_clock.cc:648] HybridClock initialized: now 1786211349530089 us; error 27 us; skew 500 ppm
Aug 08 17:49:09 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211349, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:09.532703 23707 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:49:09.533121 23707 webserver.cc:460] Webserver started at http://127.29.62.194:33875/ using document root <none> and password file <none>
I20260808 17:49:09.533342 23707 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:09.533389 23707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:09.534801 23707 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 17:49:09.535401 23730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.535581 23707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.535674 23707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "759a443ba15e42b5b6f963a44bc4ae14"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "e84809a0ec2c7296e0af083aaec134b2"
server_key_iv: "a2ef1de59fe99caecc231d641e5d335e"
server_key_version: "encryptionkey@0"
I20260808 17:49:09.536021 23707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:09.561484 23707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:09.563633 23707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:09.563804 23707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:09.564061 23707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:09.564414 23707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:09.564473 23707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.564512 23707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:09.564532 23707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.570780 23707 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:40589
I20260808 17:49:09.570833 23891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:40589 every 8 connection(s)
I20260808 17:49:09.571182 23707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:49:09.572196 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:09.571825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:42931)
0808 17:49:09.571985 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.572055 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
W20260808 17:49:09.572600 23895 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:40321 (0 consecutive failures): Network error: Failed to ping master at 127.29.62.254:40321: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
I20260808 17:49:09.572911 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:09.572825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:54023)
0808 17:49:09.572866 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.572892 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 17:49:09.573249 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:09.573159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:43009)
0808 17:49:09.573205 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.573234 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 17:49:09.573350 23895 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:49:09.576144 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 23707
I20260808 17:49:09.576421 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:40877
--local_ip_for_outbound_sockets=127.29.62.195
--tserver_master_addrs=127.29.62.254:40321
--webserver_port=33631
--webserver_interface=127.29.62.195
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:09.659106 23899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:09.659309 23899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:09.659333 23899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:09.660506 23899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:09.660549 23899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:09.660605 23899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:49:09.662173 23899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:40877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=33631
--tserver_master_addrs=127.29.62.254:40321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:09.662459 23899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:09.662698 23899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:09.667891 23906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:09.667902 23908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:09.667908 23910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:09.668265 23899 server_base.cc:1034] running on GCE node
I20260808 17:49:09.668488 23899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:09.668726 23899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:09.669898 23899 hybrid_clock.cc:648] HybridClock initialized: now 1786211349669882 us; error 33 us; skew 500 ppm
Aug 08 17:49:09 dist-test-slave-tk9z krb5kdc[21818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211349, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:09.672441 23899 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:49:09.672864 23899 webserver.cc:460] Webserver started at http://127.29.62.195:33631/ using document root <none> and password file <none>
I20260808 17:49:09.673103 23899 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:09.673175 23899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:09.674641 23899 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:09.675227 23921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.675415 23899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.675524 23899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "d896d2d1e46c41929798c81e5ce96d30"
format_stamp: "Formatted at 2026-08-08 17:48:27 on dist-test-slave-tk9z"
server_key: "a4eca87d9e24be990acff4752d76a911"
server_key_iv: "73d2a884a353066674708d20d99b7a53"
server_key_version: "encryptionkey@0"
I20260808 17:49:09.675864 23899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:09.692687 23899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:09.694625 23899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:09.694782 23899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:09.694998 23899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:09.695294 23899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:09.695349 23899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.695397 23899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:09.695446 23899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:09.701166 23899 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:40877
I20260808 17:49:09.701243 24076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:40877 every 8 connection(s)
I20260808 17:49:09.701596 23899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:49:09.702265 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 23899
I20260808 17:49:09.702889 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:09.702328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:57647)
0808 17:49:09.702565 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.702654 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":122,"threads_started":1}
W20260808 17:49:09.703265 24077 heartbeater.cc:644] Failed to heartbeat to 127.29.62.254:40321 (0 consecutive failures): Network error: Failed to ping master at 127.29.62.254:40321: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
I20260808 17:49:09.703588 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:09.703484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:53573)
0808 17:49:09.703532 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.703557 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 17:49:09.703877 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:09.703782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:55199)
0808 17:49:09.703836 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:09.703858 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 17:49:09.703990 24077 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:49:10.490932 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:10.490675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:40873)
0808 17:49:10.490809 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:10.490859 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260808 17:49:10.573948 24085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:10.573656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:41063)
0808 17:49:10.573850 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:10.573891 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":53,"threads_started":1}
I20260808 17:49:10.704493 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:10.704294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:59421)
0808 17:49:10.704412 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:10.704442 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260808 17:49:11.212301 24087 raft_consensus.cc:482] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:11.212469 24087 raft_consensus.cc:504] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:49:11.212842 24087 leader_election.cc:288] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
Aug 08 17:49:11 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211349, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.194@KRBTEST.COM
Aug 08 17:49:11 dist-test-slave-tk9z krb5kdc[21818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211349, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.195@KRBTEST.COM
I20260808 17:49:11.225404 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:11.213260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:42465 (local address 127.29.62.195:40877)
0808 17:49:11.213481 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:11.213490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:11.213509 (+ 19us) server_negotiation.cc:407] Connection header received
0808 17:49:11.213925 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:11.213927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:11.213968 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:11.214131 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:11.214958 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:11.216008 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:11.217542 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:11.218162 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:11.218281 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:11.221414 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:11.221418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:11.221419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:11.221431 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:11.223264 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:11.223911 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:11.223914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:11.223915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:11.224043 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:11.224356 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:11.224357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:11.224357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:11.224678 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:11.224767 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:11.225154 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 17:49:11.225289 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":65,"threads_started":1}
I20260808 17:49:11.225404 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:11.213289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:48059 (local address 127.29.62.194:40589)
0808 17:49:11.213446 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:11.213465 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:11.213486 (+ 21us) server_negotiation.cc:407] Connection header received
0808 17:49:11.213938 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:11.213941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:11.213977 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:11.214130 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:11.214951 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:11.216015 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:11.217543 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:11.218160 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:11.218275 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:11.221137 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:11.221141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:11.221142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:11.221153 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:11.223227 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:11.223911 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:11.223914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:11.223915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:11.224043 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:11.224356 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:11.224357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:11.224357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:11.224678 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:11.224767 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:11.225154 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 17:49:11.225289 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260808 17:49:11.225430 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:11.213154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:40877 (local address 127.29.62.193:42465)
0808 17:49:11.213275 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:11.213287 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:11.213358 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:11.214185 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:11.214187 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:11.214632 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:11.214816 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:11.214822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:11.216110 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:11.216111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:11.217441 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:11.217444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:11.218263 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:11.218264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:11.218418 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:11.219239 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:11.219243 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:11.221309 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:11.223335 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:11.223336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:11.223339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:11.223820 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:11.224121 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:11.224122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:11.224122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:11.224278 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:11.224766 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:11.224776 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:11.224935 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 17:49:11.225074 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 17:49:11.225337 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":167,"thread_start_us":49,"threads_started":1}
I20260808 17:49:11.225442 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:11.213128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:40589 (local address 127.29.62.193:48059)
0808 17:49:11.213265 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:11.213277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:11.213338 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:11.214184 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:11.214187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:11.214620 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:11.214816 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:11.214822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:11.216167 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:11.216167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:11.217441 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:11.217444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:11.218265 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:11.218265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:11.218418 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:11.219212 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:11.219223 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:11.221026 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:11.223327 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:11.223335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:11.223338 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:11.223820 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:11.224121 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:11.224122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:11.224122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:11.224278 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:11.224766 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:11.224776 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:11.224935 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 17:49:11.225074 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 17:49:11.225281 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":120,"thread_start_us":82,"threads_started":1}
W20260808 17:49:11.226876 23540 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
W20260808 17:49:11.227018 23543 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
I20260808 17:49:11.227066 23543 leader_election.cc:302] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddfb5b722d154662b946b3b884ad3722; no voters: 759a443ba15e42b5b6f963a44bc4ae14, d896d2d1e46c41929798c81e5ce96d30
I20260808 17:49:11.227208 24087 raft_consensus.cc:2726] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:49:11.493394 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:11.491505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:39193)
0808 17:49:11.493297 (+ 1792us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:11.493350 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1760}
I20260808 17:49:11.574641 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:11.574422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:54041)
0808 17:49:11.574548 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:11.574592 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260808 17:49:11.705209 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:11.704976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:34247)
0808 17:49:11.705106 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:11.705157 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260808 17:49:12.494210 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:12.493954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:42083)
0808 17:49:12.494114 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:12.494160 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260808 17:49:12.575340 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:12.575126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:34927)
0808 17:49:12.575251 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:12.575292 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260808 17:49:12.705951 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:12.705708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:59107)
0808 17:49:12.705845 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:12.705886 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260808 17:49:13.297869 24100 raft_consensus.cc:482] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:13.297967 24100 raft_consensus.cc:504] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:49:13.298141 24100 leader_election.cc:288] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
W20260808 17:49:13.298633 23540 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
W20260808 17:49:13.298748 23543 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
I20260808 17:49:13.298827 23543 leader_election.cc:302] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddfb5b722d154662b946b3b884ad3722; no voters: 759a443ba15e42b5b6f963a44bc4ae14, d896d2d1e46c41929798c81e5ce96d30
I20260808 17:49:13.298924 24100 raft_consensus.cc:2726] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:49:13.494901 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:13.494691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:37233)
0808 17:49:13.494820 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:13.494853 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260808 17:49:13.576045 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:13.575820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:51085)
0808 17:49:13.575945 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:13.575987 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260808 17:49:13.706697 24104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:13.706479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:42797)
0808 17:49:13.706600 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:13.706639 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260808 17:49:14.495707 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:14.495465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:45993)
0808 17:49:14.495620 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:14.495656 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
I20260808 17:49:14.576805 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:14.576556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:58253)
0808 17:49:14.576710 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:14.576749 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260808 17:49:14.707393 24108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:14.707182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:40331)
0808 17:49:14.707311 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:14.707351 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
I20260808 17:49:15.496506 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:15.496183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:48709)
0808 17:49:15.496400 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:15.496440 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":64,"threads_started":1}
I20260808 17:49:15.577574 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:15.577305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:55297)
0808 17:49:15.577443 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:15.577513 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260808 17:49:15.708098 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:15.707860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:35187)
0808 17:49:15.708012 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:15.708052 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260808 17:49:16.433171 24112 raft_consensus.cc:482] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:16.433279 24112 raft_consensus.cc:504] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:49:16.433574 24112 leader_election.cc:288] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
W20260808 17:49:16.433952 23543 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
W20260808 17:49:16.434087 23540 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
I20260808 17:49:16.434151 23540 leader_election.cc:302] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddfb5b722d154662b946b3b884ad3722; no voters: 759a443ba15e42b5b6f963a44bc4ae14, d896d2d1e46c41929798c81e5ce96d30
I20260808 17:49:16.434218 24112 raft_consensus.cc:2726] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:49:16.497268 24113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:16.497023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:37505)
0808 17:49:16.497174 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:16.497217 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260808 17:49:16.578276 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:16.577996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:44369)
0808 17:49:16.578169 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:16.578231 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":116,"threads_started":1}
I20260808 17:49:16.708803 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:16.708594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:58801)
0808 17:49:16.708723 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:16.708757 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260808 17:49:17.497964 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:17.497732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:43935)
0808 17:49:17.497877 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:17.497916 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260808 17:49:17.578956 24119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:17.578756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:37875)
0808 17:49:17.578875 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:17.578911 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260808 17:49:17.709447 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:17.709235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:35973)
0808 17:49:17.709364 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:17.709405 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260808 17:49:17.963546 24121 raft_consensus.cc:482] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:17.963680 24121 raft_consensus.cc:504] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:49:17.963929 24121 leader_election.cc:288] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
W20260808 17:49:17.964406 23540 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
W20260808 17:49:17.964509 23543 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
I20260808 17:49:17.964581 23543 leader_election.cc:302] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddfb5b722d154662b946b3b884ad3722; no voters: 759a443ba15e42b5b6f963a44bc4ae14, d896d2d1e46c41929798c81e5ce96d30
I20260808 17:49:17.964649 24121 raft_consensus.cc:2726] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:49:18.498756 24122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:18.498499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:49137)
0808 17:49:18.498663 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:18.498707 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260808 17:49:18.579834 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:18.579439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:34367)
0808 17:49:18.579726 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:18.579773 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":83,"threads_started":1}
I20260808 17:49:18.710173 24125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:18.709938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:52761)
0808 17:49:18.710091 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:18.710127 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260808 17:49:19.499496 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:19.499267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:38777)
0808 17:49:19.499412 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:19.499448 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260808 17:49:19.580608 24129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:19.580345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:38377)
0808 17:49:19.580506 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:19.580556 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260808 17:49:19.710939 24130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:19.710715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:59783)
0808 17:49:19.710854 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:19.710892 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260808 17:49:20.500195 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:20.499963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:41491)
0808 17:49:20.500105 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:20.500147 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260808 17:49:20.581347 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:20.581117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:49949)
0808 17:49:20.581263 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:20.581300 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260808 17:49:20.711715 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:20.711444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:34949)
0808 17:49:20.711623 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:20.711666 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
I20260808 17:49:21.500856 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:21.500653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:51933)
0808 17:49:21.500775 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:21.500808 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260808 17:49:21.581972 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:21.581777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:50957)
0808 17:49:21.581888 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:21.581921 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
I20260808 17:49:21.712433 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:21.712221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:37861)
0808 17:49:21.712355 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:21.712387 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260808 17:49:22.501686 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:22.501435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:45545)
0808 17:49:22.501590 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:22.501635 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260808 17:49:22.582679 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:22.582420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:45429)
0808 17:49:22.582584 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:22.582628 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260808 17:49:22.713050 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:22.712852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:51085)
0808 17:49:22.712971 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:22.713003 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260808 17:49:23.502449 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:23.502142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:50275)
0808 17:49:23.502288 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:23.502327 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260808 17:49:23.583329 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:23.583101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:49349)
0808 17:49:23.583242 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:23.583282 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260808 17:49:23.713678 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:23.713463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:56961)
0808 17:49:23.713591 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:23.713630 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260808 17:49:24.503213 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:24.502997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.193:53185)
0808 17:49:24.503130 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:24.503166 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260808 17:49:24.561339 24146 raft_consensus.cc:482] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:24.561441 24146 raft_consensus.cc:504] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d896d2d1e46c41929798c81e5ce96d30" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40877 } } peers { permanent_uuid: "759a443ba15e42b5b6f963a44bc4ae14" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40589 } } peers { permanent_uuid: "ddfb5b722d154662b946b3b884ad3722" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38253 } }
I20260808 17:49:24.561658 24146 leader_election.cc:288] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877), 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589)
W20260808 17:49:24.562100 23543 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d896d2d1e46c41929798c81e5ce96d30 (127.29.62.195:40877): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
W20260808 17:49:24.562196 23540 leader_election.cc:341] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 759a443ba15e42b5b6f963a44bc4ae14 (127.29.62.194:40589): Not found: Tablet not found: 42dbe43fdc0e406db696d7416032c107
I20260808 17:49:24.562240 23540 leader_election.cc:302] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddfb5b722d154662b946b3b884ad3722; no voters: 759a443ba15e42b5b6f963a44bc4ae14, d896d2d1e46c41929798c81e5ce96d30
I20260808 17:49:24.562352 24146 raft_consensus.cc:2726] T 42dbe43fdc0e406db696d7416032c107 P ddfb5b722d154662b946b3b884ad3722 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:49:24.584095 24147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:24.583851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.194:43495)
0808 17:49:24.584009 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:24.584051 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260808 17:49:24.704994 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 23474
I20260808 17:49:24.714484 24149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:24.714202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40321 (local address 127.29.62.195:37923)
0808 17:49:24.714360 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:24.714439 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.254:40321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":75,"threads_started":1}
I20260808 17:49:24.718101 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 23707
I20260808 17:49:24.726259 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 23899
2026-08-08T17:49:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59020 ms)
[----------] 30 tests from SecurityITest (239106 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:24 dist-test-slave-tk9z krb5kdc[24153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:24 dist-test-slave-tk9z krb5kdc[24153](info): set up 2 sockets
Aug 08 17:49:24 dist-test-slave-tk9z krb5kdc[24153](info): commencing operation
krb5kdc: starting...
W20260808 17:49:25.962813 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:49:26.085634 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:35947
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:35947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:26.169178 24171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:26.169395 24171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:26.169425 24171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:26.170579 24171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:26.170619 24171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:26.170634 24171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:26.170646 24171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:26.170658 24171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:26.172057 24171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:35947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:35947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:26.172248 24171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:26.172436 24171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:26.177557 24178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:26.177613 24182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:26.177631 24171 server_base.cc:1034] running on GCE node
W20260808 17:49:26.177634 24179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:26.178108 24171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:26.178448 24171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:26.179602 24171 hybrid_clock.cc:648] HybridClock initialized: now 1786211366179589 us; error 28 us; skew 500 ppm
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:26.182129 24171 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:26.182610 24171 webserver.cc:460] Webserver started at http://127.29.62.254:45465/ using document root <none> and password file <none>
I20260808 17:49:26.182837 24171 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:26.182904 24171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:26.183068 24171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:26.184137 24171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "854fe9250e22f9ff81529c613c18e2db"
server_key_iv: "334a7a5ff46ed0d7691e25dcd6522206"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.184518 24171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "854fe9250e22f9ff81529c613c18e2db"
server_key_iv: "334a7a5ff46ed0d7691e25dcd6522206"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.185858 24171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:26.186640 24194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.186812 24171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:26.186924 24171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "854fe9250e22f9ff81529c613c18e2db"
server_key_iv: "334a7a5ff46ed0d7691e25dcd6522206"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.187002 24171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:26.205236 24171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:26.207232 24171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:26.207401 24171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:26.211715 24171 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:35947
I20260808 17:49:26.211750 24260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:35947 every 8 connection(s)
I20260808 17:49:26.212143 24171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:26.213136 24261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:26.215549 24261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb: Bootstrap starting.
I20260808 17:49:26.216127 24261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:26.216363 24261 log.cc:824] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:26.217054 24261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb: No bootstrap required, opened a new log
I20260808 17:49:26.218582 24261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35947 } }
I20260808 17:49:26.218715 24261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:26.218739 24261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a30c356cc8b41e5a424a2fa99a3e4cb, State: Initialized, Role: FOLLOWER
I20260808 17:49:26.218873 24261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35947 } }
I20260808 17:49:26.218966 24261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:26.218989 24261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:26.219018 24261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:26.219481 24261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35947 } }
I20260808 17:49:26.219568 24261 leader_election.cc:302] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a30c356cc8b41e5a424a2fa99a3e4cb; no voters:
I20260808 17:49:26.219698 24261 leader_election.cc:288] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:26.219769 24264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:26.219935 24264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [term 1 LEADER]: Becoming Leader. State: Replica: 6a30c356cc8b41e5a424a2fa99a3e4cb, State: Running, Role: LEADER
I20260808 17:49:26.219969 24261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:26.220046 24264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35947 } }
I20260808 17:49:26.220422 24269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a30c356cc8b41e5a424a2fa99a3e4cb. Latest consensus state: current_term: 1 leader_uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35947 } } }
I20260808 17:49:26.220515 24269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:26.220583 24266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a30c356cc8b41e5a424a2fa99a3e4cb" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 35947 } } }
I20260808 17:49:26.220656 24266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:26.220762 24277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:26.221213 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 24171
I20260808 17:49:26.221354 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:26.221350 24277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:26.221578 29947 external_mini_cluster.cc:1428] Setting key af65c30f2408d3d5ab78b64b1632c8f1
I20260808 17:49:26.226159 24277 catalog_manager.cc:1347] Generated new cluster ID: cf00f2030769477f8d1f1c9541c336f6
I20260808 17:49:26.226217 24277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:26.232184 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.222229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43788 (local address 127.29.62.254:35947)
0808 17:49:26.222462 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.222480 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.222547 (+ 67us) server_negotiation.cc:407] Connection header received
0808 17:49:26.222697 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.222704 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.222780 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.222948 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:26.223784 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.224653 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.225765 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.226288 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.226432 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.228800 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:26.228804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:26.228804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:26.228816 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:26.230603 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.230998 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.231002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.231002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.231110 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.231333 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.231334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.231334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.231570 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:26.231739 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.231837 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.231992 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":90,"threads_started":1}
I20260808 17:49:26.240446 24277 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:26.240934 24277 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:26.243400 24277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb: Generated new TSK 0
I20260808 17:49:26.243585 24277 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:49:26.271438 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:35947
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:26.354945 24300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:26.355170 24300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:26.355212 24300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:26.356352 24300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:26.356420 24300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:26.356465 24300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:49:26.357950 24300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:26.358217 24300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:26.358503 24300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:26.363435 24307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:26.363466 24306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:26.363452 24300 server_base.cc:1034] running on GCE node
W20260808 17:49:26.363437 24309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:26.363809 24300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:26.364071 24300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:26.365257 24300 hybrid_clock.cc:648] HybridClock initialized: now 1786211366365244 us; error 29 us; skew 500 ppm
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:26.367852 24300 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:49:26.368235 24300 webserver.cc:460] Webserver started at http://127.29.62.193:35455/ using document root <none> and password file <none>
I20260808 17:49:26.368430 24300 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:26.368472 24300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:26.368566 24300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:26.369545 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "96f71a31332b4c23a86f5ea9f3bfb195"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "e24adce7805f83f3f7b3be5d1c51b5cd"
server_key_iv: "0d061d9d242e828315bd35e27d813b65"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.369849 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "96f71a31332b4c23a86f5ea9f3bfb195"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "e24adce7805f83f3f7b3be5d1c51b5cd"
server_key_iv: "0d061d9d242e828315bd35e27d813b65"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.371203 24300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:26.371869 24319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.372064 24300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:26.372138 24300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "96f71a31332b4c23a86f5ea9f3bfb195"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "e24adce7805f83f3f7b3be5d1c51b5cd"
server_key_iv: "0d061d9d242e828315bd35e27d813b65"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.372191 24300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:26.381392 24300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:26.383711 24300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:26.383822 24300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:26.384029 24300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:26.384286 24300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:26.384318 24300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.384341 24300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:26.384353 24300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.390473 24300 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:38203
I20260808 17:49:26.390488 24466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:38203 every 8 connection(s)
I20260808 17:49:26.390872 24300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:49:26.396895 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 24300
I20260808 17:49:26.397020 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:26.397248 29947 external_mini_cluster.cc:1428] Setting key c860f6cdaa75a9d9dd999477367b9fe7
I20260808 17:49:26.400746 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.391598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:50691 (local address 127.29.62.254:35947)
0808 17:49:26.391663 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.391666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.391886 (+ 220us) server_negotiation.cc:407] Connection header received
0808 17:49:26.392446 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.392447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.392463 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.392527 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:26.393500 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.394030 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.395123 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.395341 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.395412 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.397599 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:26.397601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:26.397601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:26.397610 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:26.399187 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.399702 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.399703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.399703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.399756 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.400049 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.400050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.400051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.400196 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:26.400271 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.400521 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.400622 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:49:26.400774 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.391585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:35947 (local address 127.29.62.193:50691)
0808 17:49:26.391762 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.391799 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:26.392321 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:26.392601 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:26.392605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:26.393190 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:26.393386 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.393392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.394131 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.394132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.395019 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.395021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.395442 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.395444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.395561 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.396027 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:26.396034 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:26.397512 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:26.399269 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:26.399277 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:26.399279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:26.399620 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:26.399818 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:26.399819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:26.399819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:26.399973 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:26.400267 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:26.400273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:26.400390 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 17:49:26.400492 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 17:49:26.400615 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260808 17:49:26.401647 24467 heartbeater.cc:347] Connected to a master server at 127.29.62.254:35947
I20260808 17:49:26.401754 24467 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:26.401958 24467 heartbeater.cc:511] Master 127.29.62.254:35947 requested a full tablet report, sending...
I20260808 17:49:26.402594 24216 ts_manager.cc:194] Registered new tserver with Master: 96f71a31332b4c23a86f5ea9f3bfb195 (127.29.62.193:38203)
I20260808 17:49:26.403286 24216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:50691
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:49:26.424912 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:35947
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:26.507516 24473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:26.507711 24473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:26.507735 24473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:26.508873 24473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:26.508916 24473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:26.508973 24473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:49:26.510552 24473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:26.510787 24473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:26.511016 24473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:26.516229 24484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:26.516296 24473 server_base.cc:1034] running on GCE node
W20260808 17:49:26.516238 24488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:26.516247 24485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:26.516685 24473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:26.516960 24473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:26.518182 24473 hybrid_clock.cc:648] HybridClock initialized: now 1786211366518169 us; error 30 us; skew 500 ppm
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:26.521052 24473 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:49:26.521546 24473 webserver.cc:460] Webserver started at http://127.29.62.194:35797/ using document root <none> and password file <none>
I20260808 17:49:26.521782 24473 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:26.521872 24473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:26.522043 24473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:26.523177 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "f861c98964b44e599bf411994e881786"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "4d70f9207db5b2ee7177148ac6b2efd1"
server_key_iv: "cfa0fa387187b335717f3aa8f1622be8"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.523558 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "f861c98964b44e599bf411994e881786"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "4d70f9207db5b2ee7177148ac6b2efd1"
server_key_iv: "cfa0fa387187b335717f3aa8f1622be8"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.524938 24473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:26.525674 24495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.525849 24473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:26.525965 24473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "f861c98964b44e599bf411994e881786"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "4d70f9207db5b2ee7177148ac6b2efd1"
server_key_iv: "cfa0fa387187b335717f3aa8f1622be8"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.526043 24473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:26.552431 24473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:26.554555 24473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:26.554730 24473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:26.554996 24473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:26.555320 24473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:26.555373 24473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.555410 24473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:26.555455 24473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.561318 24473 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:45859
I20260808 17:49:26.561367 24651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:45859 every 8 connection(s)
I20260808 17:49:26.561693 24473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:26.570909 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 24473
I20260808 17:49:26.571051 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:49:26.571148 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.562429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:36531 (local address 127.29.62.254:35947)
0808 17:49:26.562473 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.562475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.562695 (+ 220us) server_negotiation.cc:407] Connection header received
0808 17:49:26.563244 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.563245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.563259 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.563323 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:26.564300 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.564822 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.565824 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.566047 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.566136 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.568272 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:26.568273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:26.568273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:26.568281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:26.569679 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.570060 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.570061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.570062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.570123 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.570367 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.570428 (+ 61us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.570428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.570577 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:26.570683 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.570910 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.571019 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:49:26.571280 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.562354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:35947 (local address 127.29.62.194:36531)
0808 17:49:26.562569 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.562604 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:26.563130 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:26.563401 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:26.563409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:26.563989 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:26.564187 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.564192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.564937 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.564940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.565727 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.565729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.566148 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.566149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.566273 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.566757 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:26.566766 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:26.568182 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:26.569758 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:26.569764 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:26.569765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:26.569987 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:26.570187 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:26.570188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:26.570188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:26.570299 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:26.570640 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:26.570646 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:26.570764 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 17:49:26.570852 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 17:49:26.570997 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":92,"threads_started":1}
I20260808 17:49:26.571329 29947 external_mini_cluster.cc:1428] Setting key 675ad30a579f98c45b5d3ea0ec98c5fb
I20260808 17:49:26.571826 24652 heartbeater.cc:347] Connected to a master server at 127.29.62.254:35947
I20260808 17:49:26.571941 24652 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:26.572160 24652 heartbeater.cc:511] Master 127.29.62.254:35947 requested a full tablet report, sending...
I20260808 17:49:26.572604 24216 ts_manager.cc:194] Registered new tserver with Master: f861c98964b44e599bf411994e881786 (127.29.62.194:45859)
I20260808 17:49:26.573038 24216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:36531
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:49:26.596506 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:35947
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:26.679942 24663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:26.680163 24663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:26.680188 24663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:26.681424 24663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:26.681466 24663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:26.681493 24663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:49:26.683007 24663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:26.683262 24663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:26.683511 24663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:26.688508 24672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:26.688508 24670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:26.688534 24669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:26.688642 24663 server_base.cc:1034] running on GCE node
I20260808 17:49:26.688944 24663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:26.689193 24663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:26.690332 24663 hybrid_clock.cc:648] HybridClock initialized: now 1786211366690321 us; error 26 us; skew 500 ppm
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:26.692935 24663 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:49:26.693414 24663 webserver.cc:460] Webserver started at http://127.29.62.195:40045/ using document root <none> and password file <none>
I20260808 17:49:26.693668 24663 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:26.693715 24663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:26.693879 24663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:26.695027 24663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "79a806f831214138b9853f58b3e6ca41"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "7ee56b10db5ad0af70210c1ee4bb56d9"
server_key_iv: "4e9ab92451c3aa1d73733600cd0df57a"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.695396 24663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "79a806f831214138b9853f58b3e6ca41"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "7ee56b10db5ad0af70210c1ee4bb56d9"
server_key_iv: "4e9ab92451c3aa1d73733600cd0df57a"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.696728 24663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:49:26.697453 24684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.697638 24663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:26.697726 24663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "79a806f831214138b9853f58b3e6ca41"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "7ee56b10db5ad0af70210c1ee4bb56d9"
server_key_iv: "4e9ab92451c3aa1d73733600cd0df57a"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.697781 24663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:26.709417 24663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:26.711437 24663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:26.711558 24663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:26.711781 24663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:26.712105 24663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:26.712138 24663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.712159 24663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:26.712204 24663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.718017 24663 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:46339
I20260808 17:49:26.718086 24836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:46339 every 8 connection(s)
I20260808 17:49:26.718467 24663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:49:26.721894 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 24663
I20260808 17:49:26.722034 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:49:26.722249 29947 external_mini_cluster.cc:1428] Setting key 54cf413af170fa855a0b2634ce917cf3
Aug 08 17:49:26 dist-test-slave-tk9z krb5kdc[24153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:26.728116 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.719231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:42989 (local address 127.29.62.254:35947)
0808 17:49:26.719272 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.719274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.719475 (+ 201us) server_negotiation.cc:407] Connection header received
0808 17:49:26.720012 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.720014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.720027 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.720091 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:26.721113 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.721719 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.723162 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.723472 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.723562 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.725583 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:26.725584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:26.725584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:26.725590 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:26.727004 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.727355 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.727356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.727356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.727411 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.727618 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.727619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.727619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.727717 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:26.727766 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.727991 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.728052 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:49:26.728286 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.719190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:35947 (local address 127.29.62.195:42989)
0808 17:49:26.719332 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.719376 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:26.719906 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:26.720240 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:26.720245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:26.720823 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:26.721011 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.721016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.721853 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.721855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.723070 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.723074 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.723588 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.723589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.723726 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.724150 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:26.724162 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:26.725502 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:26.727079 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:26.727084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:26.727086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:26.727289 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:26.727470 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:26.727471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:26.727471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:26.727564 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:26.727771 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:26.727775 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:26.727880 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 17:49:26.727961 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:49:26.728087 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260808 17:49:26.728785 24837 heartbeater.cc:347] Connected to a master server at 127.29.62.254:35947
I20260808 17:49:26.728891 24837 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:26.729072 24837 heartbeater.cc:511] Master 127.29.62.254:35947 requested a full tablet report, sending...
I20260808 17:49:26.729475 24216 ts_manager.cc:194] Registered new tserver with Master: 79a806f831214138b9853f58b3e6ca41 (127.29.62.195:46339)
I20260808 17:49:26.729919 24216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:42989
I20260808 17:49:26.734074 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:49:26.741346 24295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.735465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.29.62.254:35947)
0808 17:49:26.735511 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.735512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.735569 (+ 57us) server_negotiation.cc:407] Connection header received
0808 17:49:26.735644 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.735645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.735655 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.735699 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:26.736273 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.736749 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.737481 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.737677 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.737752 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.739028 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:26.739029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:26.739029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:26.739037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:26.740176 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.740524 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.740525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.740525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.740587 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:26.740789 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:26.740790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:26.740790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:26.740947 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:26.741052 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.741165 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.741246 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:49:26.742847 24216 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:49:26.744674 24216 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:49:26.751473 24851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.747004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:38203 (local address 127.0.0.1:53514)
0808 17:49:26.747177 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.747218 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:26.747287 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:26.747622 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:26.747625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:26.747627 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:26.747696 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.747698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.748691 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.748692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.749908 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.749911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.751105 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.751107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.751189 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.751206 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 17:49:26.751275 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 17:49:26.751354 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":37,"thread_start_us":62,"threads_started":1}
I20260808 17:49:26.751518 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.747203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.29.62.194:45859)
0808 17:49:26.747394 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.747399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.747520 (+ 121us) server_negotiation.cc:407] Connection header received
0808 17:49:26.747564 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.747567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.747582 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.747656 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:26.747855 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.748972 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.750350 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.751166 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.751236 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.751294 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.751383 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.751424 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260808 17:49:26.751479 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.747233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:46339 (local address 127.0.0.1:56780)
0808 17:49:26.747448 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.747460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:26.747523 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:26.747948 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:26.747948 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:26.747951 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:26.748013 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.748014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.749286 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.749287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.750201 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.750204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.751107 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.751109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.751230 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.751236 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:49:26.751271 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 17:49:26.751352 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260808 17:49:26.751480 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.747005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45859 (local address 127.0.0.1:41996)
0808 17:49:26.747387 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.747399 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:26.747466 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:26.747705 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:26.747706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:26.747709 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:26.747769 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.747771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.749091 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.749092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.750127 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.750129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.751247 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.751248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.751305 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.751308 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:26.751366 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:49:26.751401 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":120,"threads_started":1}
I20260808 17:49:26.751770 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.747444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.29.62.195:46339)
0808 17:49:26.747634 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.747639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.747649 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:49:26.747819 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.747822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.747844 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.748093 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:26.748167 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.749178 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.750316 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.751004 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.751523 (+ 519us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.751595 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.751627 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.751676 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260808 17:49:26.752360 24852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.747142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.29.62.193:38203)
0808 17:49:26.747438 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.747442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.747453 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:49:26.747494 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.747496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.747513 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.747771 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:26.747854 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.748583 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.750233 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.750997 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.751072 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.751135 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.752224 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.752279 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":80,"threads_started":1}
I20260808 17:49:26.752472 24562 tablet_service.cc:1467] Processing CreateTablet for tablet e5d68f75538f490589c1350f14b937c6 (DEFAULT_TABLE table=test-table [id=f5974cd2885540c789dd8f744a57a6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:26.752472 24749 tablet_service.cc:1467] Processing CreateTablet for tablet e5d68f75538f490589c1350f14b937c6 (DEFAULT_TABLE table=test-table [id=f5974cd2885540c789dd8f744a57a6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:26.752858 24562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d68f75538f490589c1350f14b937c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:26.752858 24749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d68f75538f490589c1350f14b937c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:26.752976 24385 tablet_service.cc:1467] Processing CreateTablet for tablet e5d68f75538f490589c1350f14b937c6 (DEFAULT_TABLE table=test-table [id=f5974cd2885540c789dd8f744a57a6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:26.753315 24385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d68f75538f490589c1350f14b937c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:26.755890 24856 tablet_bootstrap.cc:492] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195: Bootstrap starting.
I20260808 17:49:26.756450 24857 tablet_bootstrap.cc:492] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786: Bootstrap starting.
I20260808 17:49:26.756716 24856 tablet_bootstrap.cc:654] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:26.757068 24857 tablet_bootstrap.cc:654] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:26.757301 24856 log.cc:824] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:26.757612 24857 log.cc:824] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:26.757967 24858 tablet_bootstrap.cc:492] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: Bootstrap starting.
I20260808 17:49:26.758023 24856 tablet_bootstrap.cc:492] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195: No bootstrap required, opened a new log
I20260808 17:49:26.758091 24856 ts_tablet_manager.cc:1397] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:49:26.758596 24858 tablet_bootstrap.cc:654] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:26.759068 24858 log.cc:824] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:26.759636 24857 tablet_bootstrap.cc:492] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786: No bootstrap required, opened a new log
I20260808 17:49:26.759729 24857 ts_tablet_manager.cc:1397] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 17:49:26.759904 24856 raft_consensus.cc:348] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.760082 24856 raft_consensus.cc:374] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:26.760102 24858 tablet_bootstrap.cc:492] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: No bootstrap required, opened a new log
I20260808 17:49:26.760131 24856 raft_consensus.cc:729] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f71a31332b4c23a86f5ea9f3bfb195, State: Initialized, Role: FOLLOWER
I20260808 17:49:26.760157 24858 ts_tablet_manager.cc:1397] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:49:26.760279 24856 consensus_queue.cc:260] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.760582 24856 ts_tablet_manager.cc:1428] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:49:26.760739 24467 heartbeater.cc:503] Master 127.29.62.254:35947 was elected leader, sending a full tablet report...
I20260808 17:49:26.761708 24857 raft_consensus.cc:348] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.761901 24857 raft_consensus.cc:374] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:26.761951 24857 raft_consensus.cc:729] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f861c98964b44e599bf411994e881786, State: Initialized, Role: FOLLOWER
I20260808 17:49:26.762063 24857 consensus_queue.cc:260] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.762152 24858 raft_consensus.cc:348] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.762326 24858 raft_consensus.cc:374] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:26.762341 24857 ts_tablet_manager.cc:1428] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:49:26.762367 24652 heartbeater.cc:503] Master 127.29.62.254:35947 was elected leader, sending a full tablet report...
I20260808 17:49:26.762403 24858 raft_consensus.cc:729] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79a806f831214138b9853f58b3e6ca41, State: Initialized, Role: FOLLOWER
I20260808 17:49:26.762523 24858 consensus_queue.cc:260] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.762806 24858 ts_tablet_manager.cc:1428] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:49:26.762858 24837 heartbeater.cc:503] Master 127.29.62.254:35947 was elected leader, sending a full tablet report...
I20260808 17:49:26.798439 24863 raft_consensus.cc:482] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:26.798604 24863 raft_consensus.cc:504] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.799011 24863 leader_election.cc:288] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96f71a31332b4c23a86f5ea9f3bfb195 (127.29.62.193:38203), 79a806f831214138b9853f58b3e6ca41 (127.29.62.195:46339)
I20260808 17:49:26.802325 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.799273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:43045 (local address 127.29.62.195:46339)
0808 17:49:26.799335 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.799337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.799347 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:49:26.799532 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.799533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.799544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.799612 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:26.799800 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.800336 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.801415 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.801997 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.802052 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.802060 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.802234 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.802271 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:49:26.802335 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.799202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:46339 (local address 127.29.62.194:43045)
0808 17:49:26.799299 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.799313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:26.799362 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:26.799630 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:26.799632 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:26.799634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:26.799719 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.799721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.800447 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.800448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.801340 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.801343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.802072 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.802074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.802156 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.802161 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:49:26.802221 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:49:26.802259 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 17:49:26.802788 24772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d68f75538f490589c1350f14b937c6" candidate_uuid: "f861c98964b44e599bf411994e881786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a806f831214138b9853f58b3e6ca41" is_pre_election: true
I20260808 17:49:26.802940 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.799202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:38203 (local address 127.29.62.194:36801)
0808 17:49:26.799597 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.799605 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:26.799681 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:26.799853 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:26.799854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:26.799856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:26.799956 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.799958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.800769 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.800770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.801819 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:26.801822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.802693 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.802695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:26.802762 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.802766 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:49:26.802831 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:49:26.802866 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":159,"thread_start_us":170,"threads_started":1}
I20260808 17:49:26.802971 24852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.799341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:36801 (local address 127.29.62.193:38203)
0808 17:49:26.799509 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.799511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.799669 (+ 158us) server_negotiation.cc:407] Connection header received
0808 17:49:26.799767 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.799768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.799778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.799851 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:26.800047 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.800699 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.801898 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.802460 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.802549 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.802556 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.802849 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.802885 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:49:26.802966 24772 raft_consensus.cc:2454] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f861c98964b44e599bf411994e881786 in term 0.
I20260808 17:49:26.803201 24496 leader_election.cc:302] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a806f831214138b9853f58b3e6ca41, f861c98964b44e599bf411994e881786; no voters:
I20260808 17:49:26.803267 24411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d68f75538f490589c1350f14b937c6" candidate_uuid: "f861c98964b44e599bf411994e881786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" is_pre_election: true
I20260808 17:49:26.803344 24863 raft_consensus.cc:2781] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:49:26.803396 24863 raft_consensus.cc:482] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:49:26.803426 24863 raft_consensus.cc:3037] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:26.803418 24411 raft_consensus.cc:2454] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f861c98964b44e599bf411994e881786 in term 0.
I20260808 17:49:26.804240 24863 raft_consensus.cc:504] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.804387 24863 leader_election.cc:288] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [CANDIDATE]: Term 1 election: Requested vote from peers 96f71a31332b4c23a86f5ea9f3bfb195 (127.29.62.193:38203), 79a806f831214138b9853f58b3e6ca41 (127.29.62.195:46339)
I20260808 17:49:26.804529 24411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d68f75538f490589c1350f14b937c6" candidate_uuid: "f861c98964b44e599bf411994e881786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f71a31332b4c23a86f5ea9f3bfb195"
I20260808 17:49:26.804615 24772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d68f75538f490589c1350f14b937c6" candidate_uuid: "f861c98964b44e599bf411994e881786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a806f831214138b9853f58b3e6ca41"
I20260808 17:49:26.804644 24411 raft_consensus.cc:3037] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:26.804710 24772 raft_consensus.cc:3037] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:26.805413 24772 raft_consensus.cc:2454] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f861c98964b44e599bf411994e881786 in term 1.
I20260808 17:49:26.805477 24411 raft_consensus.cc:2454] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f861c98964b44e599bf411994e881786 in term 1.
I20260808 17:49:26.805604 24496 leader_election.cc:302] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a806f831214138b9853f58b3e6ca41, f861c98964b44e599bf411994e881786; no voters:
I20260808 17:49:26.805733 24863 raft_consensus.cc:2781] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:26.805913 24863 raft_consensus.cc:686] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [term 1 LEADER]: Becoming Leader. State: Replica: f861c98964b44e599bf411994e881786, State: Running, Role: LEADER
I20260808 17:49:26.806027 24863 consensus_queue.cc:237] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } }
I20260808 17:49:26.807045 24216 catalog_manager.cc:5495] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 reported cstate change: term changed from 0 to 1, leader changed from <none> to f861c98964b44e599bf411994e881786 (127.29.62.194). New cstate: current_term: 1 leader_uuid: "f861c98964b44e599bf411994e881786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f861c98964b44e599bf411994e881786" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 } health_report { overall_health: UNKNOWN } } }
W20260808 17:49:26.812178 24657 tablet.cc:2369] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:49:26.813352 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.810943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.29.62.194:45859)
0808 17:49:26.810987 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:26.810989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:26.811000 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:49:26.811078 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:26.811079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:26.811097 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:26.811176 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:49:26.811303 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.811742 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.812578 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:26.812773 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:26.812847 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:26.813031 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:49:26.813128 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:49:26.813221 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:26.813241 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 17:49:26.813274 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:49:26.815076 24772 raft_consensus.cc:1261] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 1 FOLLOWER]: Refusing update from remote peer f861c98964b44e599bf411994e881786: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:26.815079 24411 raft_consensus.cc:1261] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 1 FOLLOWER]: Refusing update from remote peer f861c98964b44e599bf411994e881786: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:26.815438 24863 consensus_queue.cc:1035] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f71a31332b4c23a86f5ea9f3bfb195" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:26.815573 24867 consensus_queue.cc:1035] T e5d68f75538f490589c1350f14b937c6 P f861c98964b44e599bf411994e881786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a806f831214138b9853f58b3e6ca41" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:26.817693 24872 mvcc.cc:205] Tried to move back new op lower bound from 7316321758472015872 to 7316321758438338560. Current Snapshot: MvccSnapshot[applied={T|T < 7316321758472015872}]
I20260808 17:49:26.818457 24874 mvcc.cc:205] Tried to move back new op lower bound from 7316321758472015872 to 7316321758438338560. Current Snapshot: MvccSnapshot[applied={T|T < 7316321758472015872}]
I20260808 17:49:26.818614 24873 mvcc.cc:205] Tried to move back new op lower bound from 7316321758472015872 to 7316321758438338560. Current Snapshot: MvccSnapshot[applied={T|T < 7316321758472015872}]
I20260808 17:49:26.819976 24215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:26.820076 24215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:26.821090 24215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a30c356cc8b41e5a424a2fa99a3e4cb: Sending DeleteTablet for 3 replicas of tablet e5d68f75538f490589c1350f14b937c6
I20260808 17:49:26.821452 24749 tablet_service.cc:1514] Processing DeleteTablet for tablet e5d68f75538f490589c1350f14b937c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:26 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:56780
I20260808 17:49:26.821497 24562 tablet_service.cc:1514] Processing DeleteTablet for tablet e5d68f75538f490589c1350f14b937c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:26 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:41996
I20260808 17:49:26.821852 24385 tablet_service.cc:1514] Processing DeleteTablet for tablet e5d68f75538f490589c1350f14b937c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:26 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:53514
I20260808 17:49:26.822130 24886 tablet_replica.cc:330] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195: stopping tablet replica
I20260808 17:49:26.822194 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 24300
I20260808 17:49:26.822291 24886 raft_consensus.cc:2229] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:26.822458 24886 raft_consensus.cc:2258] T e5d68f75538f490589c1350f14b937c6 P 96f71a31332b4c23a86f5ea9f3bfb195 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:26.826359 24883 tablet_replica.cc:330] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: stopping tablet replica
I20260808 17:49:26.826458 24883 raft_consensus.cc:2229] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:26.826534 24883 raft_consensus.cc:2258] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:26.827536 24883 ts_tablet_manager.cc:1905] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:26.829234 24883 ts_tablet_manager.cc:1918] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:26.829311 24883 log.cc:1196] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/e5d68f75538f490589c1350f14b937c6
I20260808 17:49:26.829656 24883 ts_tablet_manager.cc:1939] T e5d68f75538f490589c1350f14b937c6 P 79a806f831214138b9853f58b3e6ca41: Deleting consensus metadata
I20260808 17:49:26.831385 24195 catalog_manager.cc:4841] TS 79a806f831214138b9853f58b3e6ca41 (127.29.62.195:46339): tablet e5d68f75538f490589c1350f14b937c6 (table test-table [id=f5974cd2885540c789dd8f744a57a6eb]) successfully deleted
W20260808 17:49:26.835235 24198 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:38203 (error 108)
I20260808 17:49:26.835516 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 24473
I20260808 17:49:26.837086 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.835687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:38203 (local address 127.0.0.1:53528)
0808 17:49:26.836995 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.837043 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:38203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1274}
W20260808 17:49:26.837560 24198 catalog_manager.cc:4584] TS 96f71a31332b4c23a86f5ea9f3bfb195 (127.29.62.193:38203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5d68f75538f490589c1350f14b937c6: Network error: Client connection negotiation failed: client connection to 127.29.62.193:38203: connect: Connection refused (error 111)
I20260808 17:49:26.843108 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 24663
I20260808 17:49:26.843133 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:26.843007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45859 (local address 127.0.0.1:42014)
0808 17:49:26.843066 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:26.843099 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.194:45859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 17:49:26.849647 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 24171
2026-08-08T17:49:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2122 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T17:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:26Z Disabled control of system clock
I20260808 17:49:26.881738 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:45423
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:45423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:26.968015 24891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:26.968245 24891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:26.968266 24891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:26.969462 24891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:26.969511 24891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:26.969523 24891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:26.969537 24891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:26.969588 24891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:26.971037 24891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:45423
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:45423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:26.971311 24891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:26.971629 24891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:26.976541 24898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:26.976632 24901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:26.976541 24899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:26.977025 24891 server_base.cc:1034] running on GCE node
I20260808 17:49:26.977272 24891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:26.977593 24891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:26.978732 24891 hybrid_clock.cc:648] HybridClock initialized: now 1786211366978709 us; error 40 us; skew 500 ppm
I20260808 17:49:26.979996 24891 webserver.cc:460] Webserver started at http://127.29.62.254:34711/ using document root <none> and password file <none>
I20260808 17:49:26.980248 24891 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:26.980295 24891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:26.980448 24891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:26.981578 24891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "d398a864b73146b7bce724ede1254941"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "d6e96d10186dfd595db6ee1f6ee0a7c4"
server_key_iv: "1968253c723fa4a91cee06270298bdf2"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.981962 24891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "d398a864b73146b7bce724ede1254941"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "d6e96d10186dfd595db6ee1f6ee0a7c4"
server_key_iv: "1968253c723fa4a91cee06270298bdf2"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.983430 24891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:26.984186 24911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:26.984370 24891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:26.984490 24891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "d398a864b73146b7bce724ede1254941"
format_stamp: "Formatted at 2026-08-08 17:49:26 on dist-test-slave-tk9z"
server_key: "d6e96d10186dfd595db6ee1f6ee0a7c4"
server_key_iv: "1968253c723fa4a91cee06270298bdf2"
server_key_version: "encryptionkey@0"
I20260808 17:49:26.984568 24891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:26.999368 24891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:26.999667 24891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:26.999821 24891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:27.003881 24891 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:45423
I20260808 17:49:27.003911 24984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:45423 every 8 connection(s)
I20260808 17:49:27.004268 24891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:27.005264 24985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:27.006572 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 24891
I20260808 17:49:27.006704 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:27.006928 29947 external_mini_cluster.cc:1428] Setting key fcc3473a3247d773779cc43544ca8dee
I20260808 17:49:27.008123 24985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941: Bootstrap starting.
I20260808 17:49:27.008796 24985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:27.009112 24985 log.cc:824] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:27.009907 24985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941: No bootstrap required, opened a new log
I20260808 17:49:27.012254 24985 raft_consensus.cc:348] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d398a864b73146b7bce724ede1254941" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45423 } }
I20260808 17:49:27.012456 24985 raft_consensus.cc:374] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:27.012519 24985 raft_consensus.cc:729] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d398a864b73146b7bce724ede1254941, State: Initialized, Role: FOLLOWER
I20260808 17:49:27.012665 24985 consensus_queue.cc:260] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d398a864b73146b7bce724ede1254941" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45423 } }
I20260808 17:49:27.012744 24985 raft_consensus.cc:388] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:27.012765 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.007469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.29.62.254:45423)
0808 17:49:27.007666 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.007674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.007698 (+ 24us) server_negotiation.cc:407] Connection header received
0808 17:49:27.008102 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.008108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.008188 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.008321 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:27.008555 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.009642 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.010807 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.011215 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.011306 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.012042 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:27.012046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:27.012046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:27.012205 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:27.012237 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:27.012247 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:27.012295 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.012402 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.012499 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260808 17:49:27.012766 24985 raft_consensus.cc:482] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:27.012883 24985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:27.013557 24985 raft_consensus.cc:504] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d398a864b73146b7bce724ede1254941" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45423 } }
I20260808 17:49:27.013679 24985 leader_election.cc:302] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d398a864b73146b7bce724ede1254941; no voters:
I20260808 17:49:27.013844 24985 leader_election.cc:288] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:27.013906 24992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:27.014055 24992 raft_consensus.cc:686] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [term 1 LEADER]: Becoming Leader. State: Replica: d398a864b73146b7bce724ede1254941, State: Running, Role: LEADER
I20260808 17:49:27.014108 24985 sys_catalog.cc:564] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:27.014148 24992 consensus_queue.cc:237] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d398a864b73146b7bce724ede1254941" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45423 } }
I20260808 17:49:27.014530 24994 sys_catalog.cc:455] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d398a864b73146b7bce724ede1254941. Latest consensus state: current_term: 1 leader_uuid: "d398a864b73146b7bce724ede1254941" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d398a864b73146b7bce724ede1254941" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45423 } } }
I20260808 17:49:27.014524 24993 sys_catalog.cc:455] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d398a864b73146b7bce724ede1254941" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d398a864b73146b7bce724ede1254941" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 45423 } } }
I20260808 17:49:27.014629 24994 sys_catalog.cc:458] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:27.014633 24993 sys_catalog.cc:458] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:27.014863 25001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:27.015348 25001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:27.016839 25001 catalog_manager.cc:1347] Generated new cluster ID: 4e661c684205466bb9d6184f697d12ce
I20260808 17:49:27.016891 25001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:27.029843 25001 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:27.030624 25001 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:27.044480 25001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941: Generated new TSK 0
I20260808 17:49:27.044692 25001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:27.055133 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45423
--builtin_ntp_servers=127.29.62.212:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:27.140493 25016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:27.140789 25016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:27.140831 25016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:27.141991 25016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:27.142060 25016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:27.142102 25016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:49:27.143507 25016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:27.143784 25016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:27.144104 25016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:27.149206 25023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:27.149217 25026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:27.149199 25024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:27.149602 25016 server_base.cc:1034] running on GCE node
I20260808 17:49:27.149796 25016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:27.149991 25016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:27.151163 25016 hybrid_clock.cc:648] HybridClock initialized: now 1786211367151149 us; error 28 us; skew 500 ppm
I20260808 17:49:27.152503 25016 webserver.cc:460] Webserver started at http://127.29.62.193:46229/ using document root <none> and password file <none>
I20260808 17:49:27.152762 25016 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:27.152810 25016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:27.152981 25016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:27.154070 25016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "32d23ba74f2c4d009319878b1160de21"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "44f89948d0ecad520638e4cbfabdc660"
server_key_iv: "6aa37bbd3726da635584cd4ef2f897a1"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.154510 25016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "32d23ba74f2c4d009319878b1160de21"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "44f89948d0ecad520638e4cbfabdc660"
server_key_iv: "6aa37bbd3726da635584cd4ef2f897a1"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.155827 25016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:27.156646 25033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.156829 25016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:27.156924 25016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "32d23ba74f2c4d009319878b1160de21"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "44f89948d0ecad520638e4cbfabdc660"
server_key_iv: "6aa37bbd3726da635584cd4ef2f897a1"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.157009 25016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:27.174893 25016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:27.175236 25016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:27.175398 25016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:27.175657 25016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:27.176012 25016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:27.176071 25016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.176108 25016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:27.176155 25016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.182327 25016 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:34217
I20260808 17:49:27.182420 25195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:34217 every 8 connection(s)
I20260808 17:49:27.182766 25016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:49:27.187928 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.183592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:34115 (local address 127.29.62.254:45423)
0808 17:49:27.183638 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.183640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.183849 (+ 209us) server_negotiation.cc:407] Connection header received
0808 17:49:27.184367 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.184368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.184379 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.184451 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:27.185105 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.185605 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.186613 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.186844 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.186899 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.187705 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:27.187707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:27.187707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:27.187711 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:27.187724 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:27.187727 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:27.187797 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.187847 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.187868 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:49:27.188155 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.183568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45423 (local address 127.29.62.193:34115)
0808 17:49:27.183747 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.183780 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:27.184272 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:27.184507 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:27.184510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:27.184518 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:27.185034 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.185038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.185686 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.185686 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.186541 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.186545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.186920 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.186921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.187038 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.187573 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:27.187573 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:27.187603 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:27.187625 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:27.187797 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:27.187799 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:27.187801 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:49:27.187846 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 17:49:27.187924 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260808 17:49:27.188651 25196 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45423
I20260808 17:49:27.188732 25196 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:27.188930 25196 heartbeater.cc:511] Master 127.29.62.254:45423 requested a full tablet report, sending...
I20260808 17:49:27.189487 24934 ts_manager.cc:194] Registered new tserver with Master: 32d23ba74f2c4d009319878b1160de21 (127.29.62.193:34217)
I20260808 17:49:27.190106 24934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.193:34115
I20260808 17:49:27.191391 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 25016
I20260808 17:49:27.191481 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:49:27.191706 29947 external_mini_cluster.cc:1428] Setting key 6ed2b362fac687782c12cee1d097ec4a
I20260808 17:49:27.192685 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45423
--builtin_ntp_servers=127.29.62.212:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:27.276623 25200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:27.276839 25200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:27.276862 25200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:27.278138 25200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:27.278182 25200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:27.278208 25200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:49:27.279661 25200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:27.279922 25200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:27.280267 25200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:27.285240 25207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:27.285250 25209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:27.285250 25206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:27.285584 25200 server_base.cc:1034] running on GCE node
I20260808 17:49:27.285828 25200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:27.286108 25200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:27.287274 25200 hybrid_clock.cc:648] HybridClock initialized: now 1786211367287268 us; error 37 us; skew 500 ppm
I20260808 17:49:27.288673 25200 webserver.cc:460] Webserver started at http://127.29.62.194:44779/ using document root <none> and password file <none>
I20260808 17:49:27.288944 25200 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:27.289016 25200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:27.289175 25200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:27.290321 25200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "d53f24fce97f4491b1401d6eb7022bd2"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "bfc1de2a50d90b3ae57dba4f385bf3b0"
server_key_iv: "743bd3cf0d8d9191ece1ab4ff9cebcf4"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.290777 25200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "d53f24fce97f4491b1401d6eb7022bd2"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "bfc1de2a50d90b3ae57dba4f385bf3b0"
server_key_iv: "743bd3cf0d8d9191ece1ab4ff9cebcf4"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.292150 25200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:27.292876 25217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.293053 25200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:27.293164 25200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "d53f24fce97f4491b1401d6eb7022bd2"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "bfc1de2a50d90b3ae57dba4f385bf3b0"
server_key_iv: "743bd3cf0d8d9191ece1ab4ff9cebcf4"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.293239 25200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:27.316097 25200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:27.316442 25200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:27.316601 25200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:27.316833 25200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:27.317200 25200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:27.317256 25200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.317301 25200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:27.317330 25200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.323047 25200 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:45499
I20260808 17:49:27.323092 25364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:45499 every 8 connection(s)
I20260808 17:49:27.323459 25200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:49:27.328460 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.324201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:48357 (local address 127.29.62.254:45423)
0808 17:49:27.324264 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.324266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.324466 (+ 200us) server_negotiation.cc:407] Connection header received
0808 17:49:27.324981 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.324982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.324993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.325045 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:27.325715 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.326212 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.327161 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.327352 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.327412 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.328175 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:27.328177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:27.328177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:27.328181 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:27.328194 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:27.328198 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:27.328241 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.328312 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.328355 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:49:27.328577 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 25200
I20260808 17:49:27.328589 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.324184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45423 (local address 127.29.62.194:48357)
0808 17:49:27.324358 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.324391 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:27.324890 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:27.325101 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:27.325105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:27.325113 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:27.325641 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.325646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.326293 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.326293 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.327084 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.327086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.327427 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.327428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.327539 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.328045 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:27.328045 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:27.328073 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:27.328096 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:27.328253 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:27.328257 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:27.328260 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:27.328307 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:49:27.328386 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260808 17:49:27.328672 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:49:27.328913 29947 external_mini_cluster.cc:1428] Setting key 95ebf4007af32110cf5790651271d99a
I20260808 17:49:27.329128 25366 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45423
I20260808 17:49:27.329228 25366 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:27.329417 25366 heartbeater.cc:511] Master 127.29.62.254:45423 requested a full tablet report, sending...
I20260808 17:49:27.329787 24934 ts_manager.cc:194] Registered new tserver with Master: d53f24fce97f4491b1401d6eb7022bd2 (127.29.62.194:45499)
I20260808 17:49:27.329953 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45423
--builtin_ntp_servers=127.29.62.212:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:49:27.330278 24934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.194:48357
W20260808 17:49:27.412874 25371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:27.413086 25371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:27.413108 25371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:27.414391 25371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:27.414433 25371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:27.414459 25371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:49:27.415884 25371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:45423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:27.416127 25371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:27.416437 25371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:27.421353 25380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:27.421366 25377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:27.421353 25376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:27.421597 25371 server_base.cc:1034] running on GCE node
I20260808 17:49:27.421801 25371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:27.422065 25371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:27.423221 25371 hybrid_clock.cc:648] HybridClock initialized: now 1786211367423219 us; error 39 us; skew 500 ppm
I20260808 17:49:27.424439 25371 webserver.cc:460] Webserver started at http://127.29.62.195:38403/ using document root <none> and password file <none>
I20260808 17:49:27.424695 25371 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:27.424765 25371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:27.424906 25371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:27.425949 25371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "19c0b498043e4f5ba9914b6c5da5ffd4"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "a219d8afcf6e888289531dd71cab8d08"
server_key_iv: "45d3729a56cc335631e558a801b0f05d"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.426326 25371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "19c0b498043e4f5ba9914b6c5da5ffd4"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "a219d8afcf6e888289531dd71cab8d08"
server_key_iv: "45d3729a56cc335631e558a801b0f05d"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.427651 25371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:27.428366 25389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.428560 25371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:49:27.428663 25371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "19c0b498043e4f5ba9914b6c5da5ffd4"
format_stamp: "Formatted at 2026-08-08 17:49:27 on dist-test-slave-tk9z"
server_key: "a219d8afcf6e888289531dd71cab8d08"
server_key_iv: "45d3729a56cc335631e558a801b0f05d"
server_key_version: "encryptionkey@0"
I20260808 17:49:27.428746 25371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:27.440659 25371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:27.440924 25371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:27.441076 25371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:27.441291 25371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:27.441614 25371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:27.441670 25371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.441704 25371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:27.441730 25371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:27.447547 25371 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:40035
I20260808 17:49:27.447584 25548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:40035 every 8 connection(s)
I20260808 17:49:27.447899 25371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:49:27.452814 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.448651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:33927 (local address 127.29.62.254:45423)
0808 17:49:27.448698 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.448700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.448920 (+ 220us) server_negotiation.cc:407] Connection header received
0808 17:49:27.449380 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.449381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.449390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.449463 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:27.450096 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.450675 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.451600 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.451784 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.451833 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.452586 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:27.452587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:27.452587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:27.452591 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:27.452604 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:27.452608 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:27.452650 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.452733 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.452755 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:49:27.453037 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.448608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:45423 (local address 127.29.62.195:33927)
0808 17:49:27.448791 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.448831 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:27.449294 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:27.449520 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:27.449523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:27.449531 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:27.450023 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.450027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.450755 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.450756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.451523 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.451525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.451857 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.451858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.451963 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.452436 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:27.452436 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:27.452464 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:27.452497 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:27.452663 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:27.452667 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:27.452669 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:49:27.452720 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:49:27.452797 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260808 17:49:27.453593 25549 heartbeater.cc:347] Connected to a master server at 127.29.62.254:45423
I20260808 17:49:27.453701 25549 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:27.453882 25549 heartbeater.cc:511] Master 127.29.62.254:45423 requested a full tablet report, sending...
I20260808 17:49:27.454254 24934 ts_manager.cc:194] Registered new tserver with Master: 19c0b498043e4f5ba9914b6c5da5ffd4 (127.29.62.195:40035)
I20260808 17:49:27.454672 24934 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.195:33927
I20260808 17:49:27.455399 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 25371
I20260808 17:49:27.455543 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:49:27.455775 29947 external_mini_cluster.cc:1428] Setting key 8833f285e544a2a8a37937fd3681a722
I20260808 17:49:27.456686 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:49:27.460786 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.458099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34324 (local address 127.29.62.254:45423)
0808 17:49:27.458171 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.458173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.458251 (+ 78us) server_negotiation.cc:407] Connection header received
0808 17:49:27.458289 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.458290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.458299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.458344 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:27.458547 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.458998 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.459689 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.459856 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.459964 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.460565 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:27.460567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:27.460567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:27.460571 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:27.460583 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:27.460587 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:27.460629 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.460678 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.460696 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 17:49:27.462291 24934 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:49:27.463804 24934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:49:27.470223 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.466259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.29.62.193:34217)
0808 17:49:27.466466 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.466471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.466483 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:49:27.466541 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.466544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.466560 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.466651 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:27.466895 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.468053 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.469336 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.469835 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.469927 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.470018 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.470102 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.470141 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":96,"threads_started":1}
I20260808 17:49:27.470211 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.465967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:34217 (local address 127.0.0.1:55534)
0808 17:49:27.466121 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.466159 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:27.466223 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:27.466725 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:27.466728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:27.466731 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:27.466797 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.466799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.468142 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.468143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.469241 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.469244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.469930 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.469931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.470008 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.470020 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 17:49:27.470088 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 17:49:27.470139 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":18,"thread_start_us":53,"threads_started":1}
I20260808 17:49:27.470407 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.465984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:40035 (local address 127.0.0.1:32864)
0808 17:49:27.466471 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.466482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:27.466582 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:27.466997 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:27.466998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:27.467000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:27.467038 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.467039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.468282 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.468283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.469182 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.469186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.470053 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.470054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.470117 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.470121 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:49:27.470154 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 17:49:27.470192 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260808 17:49:27.470556 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.466101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.29.62.195:40035)
0808 17:49:27.466507 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.466511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.466567 (+ 56us) server_negotiation.cc:407] Connection header received
0808 17:49:27.466672 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.466674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.466826 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.467112 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:27.467164 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.468187 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.469276 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.469946 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.470271 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.470392 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.470421 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.470463 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":104,"threads_started":1}
I20260808 17:49:27.470777 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.466260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.29.62.194:45499)
0808 17:49:27.466461 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.466466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.466739 (+ 273us) server_negotiation.cc:407] Connection header received
0808 17:49:27.466790 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.466792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.466808 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.466880 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:27.467064 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.467757 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.469211 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.469946 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.470030 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.470118 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.470670 (+ 552us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.470721 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":135,"threads_started":1}
I20260808 17:49:27.470924 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.465988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45499 (local address 127.0.0.1:60374)
0808 17:49:27.466676 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.466685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:27.466731 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:27.466948 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:27.466949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:27.466951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:27.466996 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.466998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.467846 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.467848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.469112 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.469115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.470150 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.470151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.470201 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.470203 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:49:27.470268 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:49:27.470538 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":29,"thread_start_us":106,"threads_started":1}
I20260808 17:49:27.471288 25459 tablet_service.cc:1467] Processing CreateTablet for tablet 0223eb240cf048ea86a89e2b9a0d01f5 (DEFAULT_TABLE table=test-table [id=672889c54da7431fab8eedd64df60a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:27.471369 25104 tablet_service.cc:1467] Processing CreateTablet for tablet 0223eb240cf048ea86a89e2b9a0d01f5 (DEFAULT_TABLE table=test-table [id=672889c54da7431fab8eedd64df60a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:27.471616 25459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0223eb240cf048ea86a89e2b9a0d01f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:27.471642 25104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0223eb240cf048ea86a89e2b9a0d01f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:27.474257 25570 tablet_bootstrap.cc:492] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21: Bootstrap starting.
I20260808 17:49:27.474287 25280 tablet_service.cc:1467] Processing CreateTablet for tablet 0223eb240cf048ea86a89e2b9a0d01f5 (DEFAULT_TABLE table=test-table [id=672889c54da7431fab8eedd64df60a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:27.474653 25280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0223eb240cf048ea86a89e2b9a0d01f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:27.474924 25570 tablet_bootstrap.cc:654] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:27.474912 25569 tablet_bootstrap.cc:492] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: Bootstrap starting.
I20260808 17:49:27.475209 25570 log.cc:824] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:27.475555 25569 tablet_bootstrap.cc:654] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:27.475868 25569 log.cc:824] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:27.475924 25570 tablet_bootstrap.cc:492] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21: No bootstrap required, opened a new log
I20260808 17:49:27.476015 25570 ts_tablet_manager.cc:1397] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:49:27.477509 25574 tablet_bootstrap.cc:492] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Bootstrap starting.
I20260808 17:49:27.478075 25574 tablet_bootstrap.cc:654] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:27.477955 25570 raft_consensus.cc:348] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.478116 25569 tablet_bootstrap.cc:492] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: No bootstrap required, opened a new log
I20260808 17:49:27.478138 25570 raft_consensus.cc:374] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:27.478168 25569 ts_tablet_manager.cc:1397] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 17:49:27.478185 25570 raft_consensus.cc:729] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d23ba74f2c4d009319878b1160de21, State: Initialized, Role: FOLLOWER
I20260808 17:49:27.478328 25570 consensus_queue.cc:260] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.478492 25574 log.cc:824] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:27.478621 25570 ts_tablet_manager.cc:1428] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:49:27.478752 25196 heartbeater.cc:503] Master 127.29.62.254:45423 was elected leader, sending a full tablet report...
I20260808 17:49:27.479573 25574 tablet_bootstrap.cc:492] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: No bootstrap required, opened a new log
I20260808 17:49:27.479653 25574 ts_tablet_manager.cc:1397] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:49:27.480542 25569 raft_consensus.cc:348] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.480662 25569 raft_consensus.cc:374] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:27.480728 25569 raft_consensus.cc:729] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c0b498043e4f5ba9914b6c5da5ffd4, State: Initialized, Role: FOLLOWER
I20260808 17:49:27.480851 25569 consensus_queue.cc:260] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.481125 25569 ts_tablet_manager.cc:1428] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:49:27.481240 25549 heartbeater.cc:503] Master 127.29.62.254:45423 was elected leader, sending a full tablet report...
I20260808 17:49:27.482034 25574 raft_consensus.cc:348] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.482172 25574 raft_consensus.cc:374] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:27.482216 25574 raft_consensus.cc:729] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53f24fce97f4491b1401d6eb7022bd2, State: Initialized, Role: FOLLOWER
I20260808 17:49:27.482331 25574 consensus_queue.cc:260] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.482638 25574 ts_tablet_manager.cc:1428] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:49:27.482653 25366 heartbeater.cc:503] Master 127.29.62.254:45423 was elected leader, sending a full tablet report...
W20260808 17:49:27.574044 25369 tablet.cc:2369] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:49:27.581847 25575 raft_consensus.cc:482] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:27.581976 25575 raft_consensus.cc:504] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.582441 25575 leader_election.cc:288] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19c0b498043e4f5ba9914b6c5da5ffd4 (127.29.62.195:40035), d53f24fce97f4491b1401d6eb7022bd2 (127.29.62.194:45499)
I20260808 17:49:27.585820 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.582622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:45499 (local address 127.29.62.193:44205)
0808 17:49:27.582699 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.582713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:27.582783 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:27.582966 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:27.582967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:27.582970 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:27.583064 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.583067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.583711 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.583712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.584889 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.584891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.585576 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.585577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.585626 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.585629 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:27.585694 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 17:49:27.585736 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 17:49:27.585824 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.582718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:44205 (local address 127.29.62.194:45499)
0808 17:49:27.582770 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.582772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.582784 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:49:27.582881 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.582882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.582892 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.582969 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:27.583159 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.583624 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.584983 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.585499 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.585574 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.585582 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.585701 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.585744 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:49:27.585980 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.582620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:40035 (local address 127.29.62.193:56135)
0808 17:49:27.582795 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.582805 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:27.582874 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:27.583040 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:27.583041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:27.583044 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:27.583128 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.583130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.583969 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.583970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.585034 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:27.585036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.585709 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.585710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:27.585773 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.585775 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:49:27.585824 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 17:49:27.585860 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260808 17:49:27.585987 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.582705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:56135 (local address 127.29.62.195:40035)
0808 17:49:27.582749 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.582751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.582878 (+ 127us) server_negotiation.cc:407] Connection header received
0808 17:49:27.582951 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.582952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.582963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.583020 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:27.583222 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.583864 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.585109 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.585629 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.585697 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.585705 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.585844 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.585901 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:49:27.586205 25302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0223eb240cf048ea86a89e2b9a0d01f5" candidate_uuid: "32d23ba74f2c4d009319878b1160de21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53f24fce97f4491b1401d6eb7022bd2" is_pre_election: true
I20260808 17:49:27.586320 25302 raft_consensus.cc:2454] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d23ba74f2c4d009319878b1160de21 in term 0.
I20260808 17:49:27.586623 25035 leader_election.cc:302] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32d23ba74f2c4d009319878b1160de21, d53f24fce97f4491b1401d6eb7022bd2; no voters:
I20260808 17:49:27.586659 25485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0223eb240cf048ea86a89e2b9a0d01f5" candidate_uuid: "32d23ba74f2c4d009319878b1160de21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" is_pre_election: true
I20260808 17:49:27.586747 25575 raft_consensus.cc:2781] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:49:27.586812 25575 raft_consensus.cc:482] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:49:27.586845 25575 raft_consensus.cc:3037] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:27.586833 25485 raft_consensus.cc:2454] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d23ba74f2c4d009319878b1160de21 in term 0.
I20260808 17:49:27.587703 25575 raft_consensus.cc:504] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.587855 25575 leader_election.cc:288] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [CANDIDATE]: Term 1 election: Requested vote from peers 19c0b498043e4f5ba9914b6c5da5ffd4 (127.29.62.195:40035), d53f24fce97f4491b1401d6eb7022bd2 (127.29.62.194:45499)
I20260808 17:49:27.587978 25485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0223eb240cf048ea86a89e2b9a0d01f5" candidate_uuid: "32d23ba74f2c4d009319878b1160de21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4"
I20260808 17:49:27.588018 25302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0223eb240cf048ea86a89e2b9a0d01f5" candidate_uuid: "32d23ba74f2c4d009319878b1160de21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53f24fce97f4491b1401d6eb7022bd2"
I20260808 17:49:27.588075 25485 raft_consensus.cc:3037] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:27.588116 25302 raft_consensus.cc:3037] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:27.588851 25485 raft_consensus.cc:2454] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d23ba74f2c4d009319878b1160de21 in term 1.
I20260808 17:49:27.588912 25302 raft_consensus.cc:2454] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d23ba74f2c4d009319878b1160de21 in term 1.
I20260808 17:49:27.589075 25039 leader_election.cc:302] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19c0b498043e4f5ba9914b6c5da5ffd4, 32d23ba74f2c4d009319878b1160de21; no voters:
I20260808 17:49:27.589185 25575 raft_consensus.cc:2781] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:27.589375 25575 raft_consensus.cc:686] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [term 1 LEADER]: Becoming Leader. State: Replica: 32d23ba74f2c4d009319878b1160de21, State: Running, Role: LEADER
I20260808 17:49:27.589481 25575 consensus_queue.cc:237] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } }
I20260808 17:49:27.590394 24934 catalog_manager.cc:5495] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32d23ba74f2c4d009319878b1160de21 (127.29.62.193). New cstate: current_term: 1 leader_uuid: "32d23ba74f2c4d009319878b1160de21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d23ba74f2c4d009319878b1160de21" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 } health_report { overall_health: UNKNOWN } } }
I20260808 17:49:27.617722 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.615472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.29.62.193:34217)
0808 17:49:27.615534 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:27.615536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:27.615545 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:49:27.615598 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:27.615599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:27.615650 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:27.615727 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:49:27.615834 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.616293 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.617010 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:27.617188 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:27.617238 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:27.617373 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:49:27.617480 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:49:27.617542 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:27.617575 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 17:49:27.617592 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:49:27.619655 25485 raft_consensus.cc:1261] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 1 FOLLOWER]: Refusing update from remote peer 32d23ba74f2c4d009319878b1160de21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:27.619655 25302 raft_consensus.cc:1261] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 1 FOLLOWER]: Refusing update from remote peer 32d23ba74f2c4d009319878b1160de21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:27.620023 25584 consensus_queue.cc:1035] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19c0b498043e4f5ba9914b6c5da5ffd4" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:27.620159 25575 consensus_queue.cc:1035] T 0223eb240cf048ea86a89e2b9a0d01f5 P 32d23ba74f2c4d009319878b1160de21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53f24fce97f4491b1401d6eb7022bd2" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:27.622799 25590 mvcc.cc:205] Tried to move back new op lower bound from 7316321761767317504 to 7316321761647280128. Current Snapshot: MvccSnapshot[applied={T|T < 7316321761767317504}]
I20260808 17:49:27.623027 25589 mvcc.cc:205] Tried to move back new op lower bound from 7316321761767317504 to 7316321761647280128. Current Snapshot: MvccSnapshot[applied={T|T < 7316321761767317504}]
I20260808 17:49:27.625114 24933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:27.625224 24933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:27.626015 24933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d398a864b73146b7bce724ede1254941: Sending DeleteTablet for 3 replicas of tablet 0223eb240cf048ea86a89e2b9a0d01f5
I20260808 17:49:27.626333 25104 tablet_service.cc:1514] Processing DeleteTablet for tablet 0223eb240cf048ea86a89e2b9a0d01f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:27 UTC) from {username='slave'} at 127.0.0.1:55534
I20260808 17:49:27.626360 25280 tablet_service.cc:1514] Processing DeleteTablet for tablet 0223eb240cf048ea86a89e2b9a0d01f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:27 UTC) from {username='slave'} at 127.0.0.1:60374
I20260808 17:49:27.626605 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 25016
I20260808 17:49:27.626716 25459 tablet_service.cc:1514] Processing DeleteTablet for tablet 0223eb240cf048ea86a89e2b9a0d01f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:27 UTC) from {username='slave'} at 127.0.0.1:32864
I20260808 17:49:27.634519 25603 tablet_replica.cc:330] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: stopping tablet replica
I20260808 17:49:27.634656 25603 raft_consensus.cc:2229] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:27.634702 25600 tablet_replica.cc:330] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: stopping tablet replica
I20260808 17:49:27.634784 25603 raft_consensus.cc:2258] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:27.634788 25600 raft_consensus.cc:2229] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:27.634877 25600 raft_consensus.cc:2258] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:27.635231 25603 ts_tablet_manager.cc:1905] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:27.635522 25600 ts_tablet_manager.cc:1905] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:27.637068 25600 ts_tablet_manager.cc:1918] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:27.637164 25600 log.cc:1196] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/0223eb240cf048ea86a89e2b9a0d01f5
I20260808 17:49:27.637251 25603 ts_tablet_manager.cc:1918] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:27.637331 25603 log.cc:1196] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/0223eb240cf048ea86a89e2b9a0d01f5
I20260808 17:49:27.637439 25600 ts_tablet_manager.cc:1939] T 0223eb240cf048ea86a89e2b9a0d01f5 P d53f24fce97f4491b1401d6eb7022bd2: Deleting consensus metadata
I20260808 17:49:27.637603 25603 ts_tablet_manager.cc:1939] T 0223eb240cf048ea86a89e2b9a0d01f5 P 19c0b498043e4f5ba9914b6c5da5ffd4: Deleting consensus metadata
I20260808 17:49:27.637881 24914 catalog_manager.cc:4841] TS d53f24fce97f4491b1401d6eb7022bd2 (127.29.62.194:45499): tablet 0223eb240cf048ea86a89e2b9a0d01f5 (table test-table [id=672889c54da7431fab8eedd64df60a36]) successfully deleted
I20260808 17:49:27.638031 24916 catalog_manager.cc:4841] TS 19c0b498043e4f5ba9914b6c5da5ffd4 (127.29.62.195:40035): tablet 0223eb240cf048ea86a89e2b9a0d01f5 (table test-table [id=672889c54da7431fab8eedd64df60a36]) successfully deleted
W20260808 17:49:27.640152 24913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:34217 (error 108)
I20260808 17:49:27.640352 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 25200
I20260808 17:49:27.642182 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:27.640538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:34217 (local address 127.0.0.1:55552)
0808 17:49:27.642090 (+ 1552us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:27.642138 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:34217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1531}
W20260808 17:49:27.642311 24913 catalog_manager.cc:4584] TS 32d23ba74f2c4d009319878b1160de21 (127.29.62.193:34217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0223eb240cf048ea86a89e2b9a0d01f5: Network error: Client connection negotiation failed: client connection to 127.29.62.193:34217: connect: Connection refused (error 111)
I20260808 17:49:27.652328 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 25371
I20260808 17:49:27.663761 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 24891
2026-08-08T17:49:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (813 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:27 dist-test-slave-tk9z krb5kdc[25609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:27 dist-test-slave-tk9z krb5kdc[25609](info): set up 2 sockets
Aug 08 17:49:27 dist-test-slave-tk9z krb5kdc[25609](info): commencing operation
krb5kdc: starting...
W20260808 17:49:28.894919 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:28 dist-test-slave-tk9z krb5kdc[25609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:49:29.016963 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:42291
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:34381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:42291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:29.098891 25627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:29.099102 25627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:29.099123 25627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:29.100255 25627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:29.100299 25627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:29.100314 25627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:29.100353 25627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:29.100383 25627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:29.101814 25627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:42291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:42291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:29.102066 25627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:29.102300 25627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:29.107180 25635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:29.107206 25638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:29.107257 25627 server_base.cc:1034] running on GCE node
W20260808 17:49:29.107225 25634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:29.107578 25627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:29.107903 25627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:29.109071 25627 hybrid_clock.cc:648] HybridClock initialized: now 1786211369109066 us; error 37 us; skew 500 ppm
Aug 08 17:49:29 dist-test-slave-tk9z krb5kdc[25609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:29.111707 25627 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:29.112089 25627 webserver.cc:460] Webserver started at http://127.29.62.254:33371/ using document root <none> and password file <none>
I20260808 17:49:29.112264 25627 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:29.112303 25627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:29.112392 25627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:29.113389 25627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "a3274117adbc4b8391a7fb38e3d68cd3"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "1cca76d4d2a5b37f47267043f04e1ead"
server_key_iv: "e583768a60303cdcd3fb3b36fc2676b8"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.113765 25627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "a3274117adbc4b8391a7fb38e3d68cd3"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "1cca76d4d2a5b37f47267043f04e1ead"
server_key_iv: "e583768a60303cdcd3fb3b36fc2676b8"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.115185 25627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 17:49:29.115909 25653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.116093 25627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:29.116189 25627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "a3274117adbc4b8391a7fb38e3d68cd3"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "1cca76d4d2a5b37f47267043f04e1ead"
server_key_iv: "e583768a60303cdcd3fb3b36fc2676b8"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.116262 25627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:29.139413 25627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:29.141494 25627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:29.141659 25627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:29.145917 25627 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:42291
I20260808 17:49:29.145982 25719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:42291 every 8 connection(s)
I20260808 17:49:29.146364 25627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:29.147367 25721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:29.149665 25721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3: Bootstrap starting.
I20260808 17:49:29.150208 25721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:29.150548 25721 log.cc:824] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:29.151228 25721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3: No bootstrap required, opened a new log
I20260808 17:49:29.151719 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 25627
I20260808 17:49:29.151844 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:29.152033 29947 external_mini_cluster.cc:1428] Setting key 36e05cfef88f99556d0c5a69da643487
I20260808 17:49:29.153271 25721 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3274117adbc4b8391a7fb38e3d68cd3" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42291 } }
I20260808 17:49:29.153450 25721 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:29.153492 25721 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3274117adbc4b8391a7fb38e3d68cd3, State: Initialized, Role: FOLLOWER
I20260808 17:49:29.153633 25721 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3274117adbc4b8391a7fb38e3d68cd3" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42291 } }
I20260808 17:49:29.153723 25721 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:29.153781 25721 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:29.153833 25721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:29.154887 25721 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3274117adbc4b8391a7fb38e3d68cd3" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42291 } }
I20260808 17:49:29.155016 25721 leader_election.cc:302] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3274117adbc4b8391a7fb38e3d68cd3; no voters:
I20260808 17:49:29.155234 25721 leader_election.cc:288] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:29.155265 25730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:29.155375 25730 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [term 1 LEADER]: Becoming Leader. State: Replica: a3274117adbc4b8391a7fb38e3d68cd3, State: Running, Role: LEADER
I20260808 17:49:29.155514 25721 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:29.155519 25730 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3274117adbc4b8391a7fb38e3d68cd3" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42291 } }
I20260808 17:49:29.155977 25732 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3274117adbc4b8391a7fb38e3d68cd3. Latest consensus state: current_term: 1 leader_uuid: "a3274117adbc4b8391a7fb38e3d68cd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3274117adbc4b8391a7fb38e3d68cd3" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42291 } } }
I20260808 17:49:29.156087 25732 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:29.156273 25731 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3274117adbc4b8391a7fb38e3d68cd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3274117adbc4b8391a7fb38e3d68cd3" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 42291 } } }
I20260808 17:49:29.156366 25731 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3 [sys.catalog]: This master's current role is: LEADER
W20260808 17:49:29.157377 25750 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:49:29.157439 25750 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:49:29.157503 25740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:29.157665 25740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:29.159142 25740 catalog_manager.cc:1347] Generated new cluster ID: d09bcb74b2b0487ba06c364b6d2c2bfd
I20260808 17:49:29.159200 25740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:49:29 dist-test-slave-tk9z krb5kdc[25609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:29.163470 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.152574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.29.62.254:42291)
0808 17:49:29.152782 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.152789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.152813 (+ 24us) server_negotiation.cc:407] Connection header received
0808 17:49:29.153251 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.153257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.153309 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.153417 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:29.154292 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.155340 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.156462 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.157192 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.157312 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.159989 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:29.159993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:29.159994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:29.160006 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:29.161687 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.162070 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.162073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.162074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.162195 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.162456 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.162457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.162458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.162693 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:29.162864 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.162959 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.163102 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260808 17:49:29.165624 25740 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:29.166299 25740 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:29.173803 25740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3: Generated new TSK 0
I20260808 17:49:29.173960 25740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:49:29.200543 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42291
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:34381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:29.283708 25756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:29.283923 25756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:29.283948 25756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:29.285123 25756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:29.285169 25756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:29.285197 25756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:49:29.286825 25756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:29.287089 25756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:29.287350 25756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:29.292513 25767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:29.292557 25764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:29.292532 25756 server_base.cc:1034] running on GCE node
W20260808 17:49:29.292570 25765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:29.292994 25756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:29.293196 25756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:29.294335 25756 hybrid_clock.cc:648] HybridClock initialized: now 1786211369294322 us; error 28 us; skew 500 ppm
Aug 08 17:49:29 dist-test-slave-tk9z krb5kdc[25609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:29.296866 25756 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:49:29.297264 25756 webserver.cc:460] Webserver started at http://127.29.62.193:40223/ using document root <none> and password file <none>
I20260808 17:49:29.297447 25756 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:29.297503 25756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:29.297595 25756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:29.298705 25756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "80374ad56c92453e8abb5e1a971f2ddb"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "5ea84f02bcd582fb2d1b5ecc454379f2"
server_key_iv: "362ed8945ff40f676259305d5cf1d11f"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.299022 25756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "80374ad56c92453e8abb5e1a971f2ddb"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "5ea84f02bcd582fb2d1b5ecc454379f2"
server_key_iv: "362ed8945ff40f676259305d5cf1d11f"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.300273 25756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:29.300967 25775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.301160 25756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:29.301249 25756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "80374ad56c92453e8abb5e1a971f2ddb"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "5ea84f02bcd582fb2d1b5ecc454379f2"
server_key_iv: "362ed8945ff40f676259305d5cf1d11f"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.301303 25756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:29.314080 25756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:29.316011 25756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:29.316119 25756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:29.316287 25756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:29.316604 25756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:29.316638 25756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.316692 25756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:29.316720 25756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.322625 25756 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:35889
I20260808 17:49:29.322793 25922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:35889 every 8 connection(s)
I20260808 17:49:29.323045 25756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:49:29.326014 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 25756
I20260808 17:49:29.326148 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:49:29.326354 29947 external_mini_cluster.cc:1428] Setting key 7482652896ffa8d1073174e66f6953d8
Aug 08 17:49:29 dist-test-slave-tk9z krb5kdc[25609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:29.333837 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.323919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42291 (local address 127.29.62.193:39707)
0808 17:49:29.324107 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.324158 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:29.324687 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:29.324903 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:29.324907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:29.325532 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:29.325735 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.325740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.326812 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.326813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.327840 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.327843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.328454 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.328455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.328578 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.329131 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:29.329140 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:29.330648 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:29.332532 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:29.332539 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:29.332540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:29.332748 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:29.332963 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:29.332964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:29.332964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:29.333076 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:29.333373 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:29.333377 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:29.333487 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 17:49:29.333567 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 17:49:29.333684 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260808 17:49:29.333860 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.323949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:39707 (local address 127.29.62.254:42291)
0808 17:49:29.324025 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.324027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.324249 (+ 222us) server_negotiation.cc:407] Connection header received
0808 17:49:29.324775 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.324776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.324789 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.324838 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:29.325838 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.326682 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.327938 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.328269 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.328344 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.330741 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:29.330743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:29.330743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:29.330755 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:29.332450 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.332826 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.332827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.332827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.332891 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.333146 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.333147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.333147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.333301 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:29.333382 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.333623 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.333726 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 17:49:29.334506 25924 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42291
I20260808 17:49:29.334610 25924 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:29.334802 25924 heartbeater.cc:511] Master 127.29.62.254:42291 requested a full tablet report, sending...
I20260808 17:49:29.335430 25675 ts_manager.cc:194] Registered new tserver with Master: 80374ad56c92453e8abb5e1a971f2ddb (127.29.62.193:35889)
I20260808 17:49:29.336371 25675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:39707
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:49:29.353564 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42291
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:34381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:29.441351 25934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:29.441603 25934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:29.441644 25934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:29.442814 25934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:29.442881 25934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:29.442934 25934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:49:29.444406 25934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:29.444671 25934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:29.444913 25934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:29.449978 25945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:29.449949 25942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:29.449954 25943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:29.450240 25934 server_base.cc:1034] running on GCE node
I20260808 17:49:29.450515 25934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:29.450778 25934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:29.451938 25934 hybrid_clock.cc:648] HybridClock initialized: now 1786211369451925 us; error 24 us; skew 500 ppm
Aug 08 17:49:29 dist-test-slave-tk9z krb5kdc[25609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:29.454622 25934 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:49:29.455060 25934 webserver.cc:460] Webserver started at http://127.29.62.194:42901/ using document root <none> and password file <none>
I20260808 17:49:29.455294 25934 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:29.455363 25934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:29.455515 25934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:29.456664 25934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "d7e3e5a3ddb34b68b483f6ad290d6410"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "3f41584e0fc21fa1ae198ef7acb1defa"
server_key_iv: "d4892696aede87c804052beb936bce2a"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.457108 25934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "d7e3e5a3ddb34b68b483f6ad290d6410"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "3f41584e0fc21fa1ae198ef7acb1defa"
server_key_iv: "d4892696aede87c804052beb936bce2a"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.458524 25934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:49:29.459311 25954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.459506 25934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:29.459612 25934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "d7e3e5a3ddb34b68b483f6ad290d6410"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "3f41584e0fc21fa1ae198ef7acb1defa"
server_key_iv: "d4892696aede87c804052beb936bce2a"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.459702 25934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:29.496097 25934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:29.498665 25934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:29.498837 25934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:29.499111 25934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:29.499444 25934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:29.499511 25934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.499557 25934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:29.499584 25934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.505666 25934 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:39653
I20260808 17:49:29.505777 26112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:39653 every 8 connection(s)
I20260808 17:49:29.506090 25934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:49:29.509960 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 25934
I20260808 17:49:29.510103 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:49:29.510344 29947 external_mini_cluster.cc:1428] Setting key 156b726425e8358b8433a4dd869bf4d0
Aug 08 17:49:29 dist-test-slave-tk9z krb5kdc[25609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:29.516302 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.506894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:58929 (local address 127.29.62.254:42291)
0808 17:49:29.506960 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.506962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.507147 (+ 185us) server_negotiation.cc:407] Connection header received
0808 17:49:29.507644 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.507645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.507655 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.507716 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:29.508635 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.509128 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.510320 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.510595 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.510654 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.513151 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:29.513153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:29.513153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:29.513162 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:29.514709 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.515222 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.515223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.515224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.515276 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.515566 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.515567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.515567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.515668 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:29.515719 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.516090 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.516184 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:49:29.516577 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.506863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42291 (local address 127.29.62.194:58929)
0808 17:49:29.507020 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.507060 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:29.507557 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:29.507771 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:29.507774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:29.508347 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:29.508558 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.508562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.509215 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.509215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.510220 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.510223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.510682 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.510683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.510807 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.511320 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:29.511328 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:29.513041 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:29.514794 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:29.514801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:29.514803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:29.515141 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:29.515339 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:29.515340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:29.515340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:29.515500 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:29.515734 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:29.515742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:29.515942 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 17:49:29.516073 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 17:49:29.516266 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260808 17:49:29.517153 26113 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42291
I20260808 17:49:29.517254 26113 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:29.517460 26113 heartbeater.cc:511] Master 127.29.62.254:42291 requested a full tablet report, sending...
I20260808 17:49:29.517884 25675 ts_manager.cc:194] Registered new tserver with Master: d7e3e5a3ddb34b68b483f6ad290d6410 (127.29.62.194:39653)
I20260808 17:49:29.518286 25675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:58929
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:49:29.538303 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42291
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:34381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:29.621412 26119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:29.621654 26119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:29.621697 26119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:29.622913 26119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:29.622987 26119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:29.623034 26119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:49:29.624545 26119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:34381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:42291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:29.624833 26119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:29.625082 26119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:29.630117 26128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:29.630182 26119 server_base.cc:1034] running on GCE node
W20260808 17:49:29.630126 26130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:29.630128 26133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:29.630554 26119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:29.630838 26119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:29.632004 26119 hybrid_clock.cc:648] HybridClock initialized: now 1786211369632004 us; error 43 us; skew 500 ppm
Aug 08 17:49:29 dist-test-slave-tk9z krb5kdc[25609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:29.634620 26119 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:49:29.635074 26119 webserver.cc:460] Webserver started at http://127.29.62.195:37289/ using document root <none> and password file <none>
I20260808 17:49:29.635269 26119 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:29.635310 26119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:29.635399 26119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:29.636379 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "764a3ed9d24c4db683e449c18fef73d9"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "28096e6239091fd19914fb52126db62b"
server_key_iv: "341b170256a802863163bb12d09ef92a"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.636682 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "764a3ed9d24c4db683e449c18fef73d9"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "28096e6239091fd19914fb52126db62b"
server_key_iv: "341b170256a802863163bb12d09ef92a"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.637952 26119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:29.638665 26140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.638833 26119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:49:29.638922 26119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "764a3ed9d24c4db683e449c18fef73d9"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "28096e6239091fd19914fb52126db62b"
server_key_iv: "341b170256a802863163bb12d09ef92a"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.638978 26119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:29.647471 26119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:29.649317 26119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:29.649435 26119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:29.649643 26119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:29.649971 26119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:29.650007 26119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.650030 26119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:29.650043 26119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.656054 26119 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:37641
I20260808 17:49:29.656145 26291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:37641 every 8 connection(s)
I20260808 17:49:29.656566 26119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:49:29 dist-test-slave-tk9z krb5kdc[25609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211369, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:29.663622 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 26119
I20260808 17:49:29.663753 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:49:29.663996 29947 external_mini_cluster.cc:1428] Setting key 02234448132335fbb33ed17838479c01
I20260808 17:49:29.666167 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.657320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:45745 (local address 127.29.62.254:42291)
0808 17:49:29.657384 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.657386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.657606 (+ 220us) server_negotiation.cc:407] Connection header received
0808 17:49:29.658088 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.658089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.658100 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.658153 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:29.659148 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.659644 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.660595 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.660775 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.660827 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.663056 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:29.663057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:29.663058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:29.663065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:29.664715 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.665160 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.665161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.665162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.665226 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.665501 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.665501 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.665502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.665627 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:29.665699 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.665995 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.666075 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:49:29.666424 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.657319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:42291 (local address 127.29.62.195:45745)
0808 17:49:29.657479 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.657515 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:29.658002 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:29.658209 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:29.658212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:29.658864 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:29.659068 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.659074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.659725 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.659726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.660522 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.660524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.660885 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.660887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.661014 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.661470 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:29.661478 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:29.662964 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:29.664817 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:29.664825 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:29.664827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:29.665063 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:29.665300 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:29.665301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:29.665302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:29.665413 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:29.665701 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:29.665709 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:29.665854 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 17:49:29.665958 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 17:49:29.666125 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260808 17:49:29.667109 26292 heartbeater.cc:347] Connected to a master server at 127.29.62.254:42291
I20260808 17:49:29.667196 26292 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:29.667371 26292 heartbeater.cc:511] Master 127.29.62.254:42291 requested a full tablet report, sending...
I20260808 17:49:29.667683 25675 ts_manager.cc:194] Registered new tserver with Master: 764a3ed9d24c4db683e449c18fef73d9 (127.29.62.195:37641)
I20260808 17:49:29.668066 25675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:45745
I20260808 17:49:29.675722 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:49:29.683682 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.677097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.29.62.254:42291)
0808 17:49:29.677168 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.677170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.677269 (+ 99us) server_negotiation.cc:407] Connection header received
0808 17:49:29.677348 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.677348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.677358 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.677414 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:29.677962 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.678522 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.679243 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.679445 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.679533 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.680921 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:29.680923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:29.680923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:29.680930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:29.682351 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.682838 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.682839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.682839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.682902 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:29.683129 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:29.683130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:29.683130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:29.683270 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:29.683385 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.683516 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.683581 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:49:29.685415 25675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:49:29.687134 25675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:49:29.694123 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.689446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37641 (local address 127.0.0.1:45704)
0808 17:49:29.689990 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.689999 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:29.690482 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:29.690759 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:29.690760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:29.690762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:29.690821 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.690822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.692043 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.692044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.692969 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.692979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.693824 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.693825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.693913 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.693927 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 17:49:29.693993 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:49:29.694043 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":2,"thread_start_us":52,"threads_started":1}
I20260808 17:49:29.694300 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.689445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:39653 (local address 127.0.0.1:45210)
0808 17:49:29.689775 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.689851 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:29.689966 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:29.690519 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:29.690523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:29.690525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:29.690593 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.690596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.691970 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.691971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.693346 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.693348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.694046 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.694047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.694112 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.694116 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:49:29.694183 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 17:49:29.694225 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":102,"thread_start_us":108,"threads_started":1}
I20260808 17:49:29.694319 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.689566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45210 (local address 127.29.62.194:39653)
0808 17:49:29.689881 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.689887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.689926 (+ 39us) server_negotiation.cc:407] Connection header received
0808 17:49:29.690101 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.690104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.690121 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.690209 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:29.690691 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.691833 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.693458 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.693934 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.694040 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.694097 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.694197 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.694231 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":79,"threads_started":1}
I20260808 17:49:29.694548 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.689530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.29.62.195:37641)
0808 17:49:29.690070 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.690073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.690087 (+ 14us) server_negotiation.cc:407] Connection header received
0808 17:49:29.690583 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.690586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.690602 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.690675 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:29.690900 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.691946 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.693071 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.693666 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.693735 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.693788 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.694314 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.694364 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":104,"threads_started":1}
I20260808 17:49:29.695061 26022 tablet_service.cc:1467] Processing CreateTablet for tablet 86f549414d8841629f16ab9cdccfa329 (DEFAULT_TABLE table=test-table [id=297cc796fa0c4720aec313d0194e1bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:29.695127 26205 tablet_service.cc:1467] Processing CreateTablet for tablet 86f549414d8841629f16ab9cdccfa329 (DEFAULT_TABLE table=test-table [id=297cc796fa0c4720aec313d0194e1bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:29.695361 26022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f549414d8841629f16ab9cdccfa329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:29.695391 26205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f549414d8841629f16ab9cdccfa329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:29.697854 26313 tablet_bootstrap.cc:492] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: Bootstrap starting.
I20260808 17:49:29.697916 26314 tablet_bootstrap.cc:492] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410: Bootstrap starting.
I20260808 17:49:29.698568 26313 tablet_bootstrap.cc:654] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:29.698625 26314 tablet_bootstrap.cc:654] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:29.698838 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.689644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:35889 (local address 127.0.0.1:39728)
0808 17:49:29.690335 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.690344 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:29.690424 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:29.690939 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:29.690940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:29.690942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:29.690999 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.691000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.691876 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.691877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.693325 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.693329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.698562 (+ 5233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.698564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.698631 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.698636 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:49:29.698693 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 17:49:29.698746 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":103,"thread_start_us":69,"threads_started":1}
I20260808 17:49:29.699091 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.689767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39728 (local address 127.29.62.193:35889)
0808 17:49:29.689969 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.689973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.690432 (+ 459us) server_negotiation.cc:407] Connection header received
0808 17:49:29.690622 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.690625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.690695 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.690821 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:29.691077 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.691773 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.693422 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.694137 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.698542 (+ 4405us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.698708 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.698746 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.698802 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260808 17:49:29.699187 26314 log.cc:824] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:29.699187 26313 log.cc:824] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:29.699703 25840 tablet_service.cc:1467] Processing CreateTablet for tablet 86f549414d8841629f16ab9cdccfa329 (DEFAULT_TABLE table=test-table [id=297cc796fa0c4720aec313d0194e1bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:29.700021 26313 tablet_bootstrap.cc:492] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: No bootstrap required, opened a new log
I20260808 17:49:29.700047 25840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86f549414d8841629f16ab9cdccfa329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:29.700095 26313 ts_tablet_manager.cc:1397] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:49:29.702128 26313 raft_consensus.cc:348] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.702325 26313 raft_consensus.cc:374] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:29.702400 26313 raft_consensus.cc:729] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 764a3ed9d24c4db683e449c18fef73d9, State: Initialized, Role: FOLLOWER
I20260808 17:49:29.702539 26313 consensus_queue.cc:260] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.702839 26313 ts_tablet_manager.cc:1428] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 17:49:29.702858 26292 heartbeater.cc:503] Master 127.29.62.254:42291 was elected leader, sending a full tablet report...
I20260808 17:49:29.703269 26314 tablet_bootstrap.cc:492] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410: No bootstrap required, opened a new log
I20260808 17:49:29.703338 26314 ts_tablet_manager.cc:1397] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 17:49:29.703348 26319 tablet_bootstrap.cc:492] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb: Bootstrap starting.
I20260808 17:49:29.704270 26319 tablet_bootstrap.cc:654] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:29.704816 26319 log.cc:824] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:29.705204 26314 raft_consensus.cc:348] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.705456 26314 raft_consensus.cc:374] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:29.705552 26314 raft_consensus.cc:729] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e3e5a3ddb34b68b483f6ad290d6410, State: Initialized, Role: FOLLOWER
I20260808 17:49:29.705621 26319 tablet_bootstrap.cc:492] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb: No bootstrap required, opened a new log
I20260808 17:49:29.705684 26319 ts_tablet_manager.cc:1397] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:49:29.705675 26314 consensus_queue.cc:260] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.705945 26314 ts_tablet_manager.cc:1428] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:49:29.705986 26113 heartbeater.cc:503] Master 127.29.62.254:42291 was elected leader, sending a full tablet report...
I20260808 17:49:29.707511 26319 raft_consensus.cc:348] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.707638 26319 raft_consensus.cc:374] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:29.707659 26319 raft_consensus.cc:729] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80374ad56c92453e8abb5e1a971f2ddb, State: Initialized, Role: FOLLOWER
I20260808 17:49:29.707731 26319 consensus_queue.cc:260] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.707973 26319 ts_tablet_manager.cc:1428] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:49:29.708006 25924 heartbeater.cc:503] Master 127.29.62.254:42291 was elected leader, sending a full tablet report...
I20260808 17:49:29.725044 26323 raft_consensus.cc:482] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:29.725135 26323 raft_consensus.cc:504] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.725524 26323 leader_election.cc:288] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 764a3ed9d24c4db683e449c18fef73d9 (127.29.62.195:37641), d7e3e5a3ddb34b68b483f6ad290d6410 (127.29.62.194:39653)
I20260808 17:49:29.729380 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.725842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:48527 (local address 127.29.62.194:39653)
0808 17:49:29.725900 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.725902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.726035 (+ 133us) server_negotiation.cc:407] Connection header received
0808 17:49:29.726072 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.726073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.726089 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.726157 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:29.726277 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.726989 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.728455 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.729016 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.729095 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.729103 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.729272 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.729310 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:49:29.729380 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.725818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:45141 (local address 127.29.62.195:37641)
0808 17:49:29.725869 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.725871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.725882 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:49:29.725911 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.725912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.725922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.725990 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:29.726160 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.726947 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.728456 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.729023 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.729094 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.729103 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.729272 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.729309 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:49:29.729409 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.725696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:37641 (local address 127.29.62.193:45141)
0808 17:49:29.725759 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.725772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:29.725823 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:29.726006 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:29.726007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:29.726009 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:29.726083 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.726084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.727044 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.727045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.728360 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.728363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.729103 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.729104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.729188 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.729190 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:49:29.729247 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 17:49:29.729293 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":51}
I20260808 17:49:29.729415 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.725698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:39653 (local address 127.29.62.193:48527)
0808 17:49:29.725946 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.725954 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:29.726023 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:29.726163 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:29.726164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:29.726165 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:29.726219 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.726220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.727061 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.727062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.728360 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:29.728363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.729104 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.729104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:29.729184 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.729189 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:49:29.729247 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:49:29.729293 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":94,"thread_start_us":106,"threads_started":1}
I20260808 17:49:29.729782 26229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f549414d8841629f16ab9cdccfa329" candidate_uuid: "80374ad56c92453e8abb5e1a971f2ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "764a3ed9d24c4db683e449c18fef73d9" is_pre_election: true
I20260808 17:49:29.729950 26229 raft_consensus.cc:2454] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80374ad56c92453e8abb5e1a971f2ddb in term 0.
I20260808 17:49:29.730052 26050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f549414d8841629f16ab9cdccfa329" candidate_uuid: "80374ad56c92453e8abb5e1a971f2ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" is_pre_election: true
I20260808 17:49:29.730206 26050 raft_consensus.cc:2454] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80374ad56c92453e8abb5e1a971f2ddb in term 0.
I20260808 17:49:29.730239 25778 leader_election.cc:302] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 764a3ed9d24c4db683e449c18fef73d9, 80374ad56c92453e8abb5e1a971f2ddb; no voters:
I20260808 17:49:29.730422 26323 raft_consensus.cc:2781] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:49:29.730480 26323 raft_consensus.cc:482] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:49:29.730499 26323 raft_consensus.cc:3037] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:29.731212 26323 raft_consensus.cc:504] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.731354 26323 leader_election.cc:288] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [CANDIDATE]: Term 1 election: Requested vote from peers 764a3ed9d24c4db683e449c18fef73d9 (127.29.62.195:37641), d7e3e5a3ddb34b68b483f6ad290d6410 (127.29.62.194:39653)
I20260808 17:49:29.731501 26229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f549414d8841629f16ab9cdccfa329" candidate_uuid: "80374ad56c92453e8abb5e1a971f2ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "764a3ed9d24c4db683e449c18fef73d9"
I20260808 17:49:29.731585 26229 raft_consensus.cc:3037] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:29.732272 26229 raft_consensus.cc:2454] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80374ad56c92453e8abb5e1a971f2ddb in term 1.
I20260808 17:49:29.732401 26050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86f549414d8841629f16ab9cdccfa329" candidate_uuid: "80374ad56c92453e8abb5e1a971f2ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410"
I20260808 17:49:29.732479 25778 leader_election.cc:302] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 764a3ed9d24c4db683e449c18fef73d9, 80374ad56c92453e8abb5e1a971f2ddb; no voters:
I20260808 17:49:29.732488 26050 raft_consensus.cc:3037] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:29.732554 26323 raft_consensus.cc:2781] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:29.732709 26323 raft_consensus.cc:686] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [term 1 LEADER]: Becoming Leader. State: Replica: 80374ad56c92453e8abb5e1a971f2ddb, State: Running, Role: LEADER
I20260808 17:49:29.732806 26323 consensus_queue.cc:237] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } }
I20260808 17:49:29.733402 26050 raft_consensus.cc:2454] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80374ad56c92453e8abb5e1a971f2ddb in term 1.
I20260808 17:49:29.733598 25675 catalog_manager.cc:5495] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80374ad56c92453e8abb5e1a971f2ddb (127.29.62.193). New cstate: current_term: 1 leader_uuid: "80374ad56c92453e8abb5e1a971f2ddb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80374ad56c92453e8abb5e1a971f2ddb" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 35889 } health_report { overall_health: HEALTHY } } }
I20260808 17:49:29.738588 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.736249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39738 (local address 127.29.62.193:35889)
0808 17:49:29.736299 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.736301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.736310 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:49:29.736385 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.736386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.736408 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.736498 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:49:29.736628 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.737071 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.737815 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.738003 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.738080 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.738232 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:49:29.738323 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:49:29.738464 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.738512 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.738524 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:49:29.740203 26229 raft_consensus.cc:1261] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 1 FOLLOWER]: Refusing update from remote peer 80374ad56c92453e8abb5e1a971f2ddb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:29.740223 26050 raft_consensus.cc:1261] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 1 FOLLOWER]: Refusing update from remote peer 80374ad56c92453e8abb5e1a971f2ddb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:29.740540 26323 consensus_queue.cc:1035] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e3e5a3ddb34b68b483f6ad290d6410" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 39653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:29.740682 26325 consensus_queue.cc:1035] T 86f549414d8841629f16ab9cdccfa329 P 80374ad56c92453e8abb5e1a971f2ddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "764a3ed9d24c4db683e449c18fef73d9" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 37641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:29.744041 26328 mvcc.cc:205] Tried to move back new op lower bound from 7316321770453495808 to 7316321770426183680. Current Snapshot: MvccSnapshot[applied={T|T < 7316321770453495808 or (T in {7316321770453495808})}]
I20260808 17:49:29.745159 25675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:29.745249 25675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:29.745932 25675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3274117adbc4b8391a7fb38e3d68cd3: Sending DeleteTablet for 3 replicas of tablet 86f549414d8841629f16ab9cdccfa329
I20260808 17:49:29.746268 26205 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f549414d8841629f16ab9cdccfa329 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:29 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45704
I20260808 17:49:29.746318 25840 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f549414d8841629f16ab9cdccfa329 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:29 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:39728
I20260808 17:49:29.746523 26022 tablet_service.cc:1514] Processing DeleteTablet for tablet 86f549414d8841629f16ab9cdccfa329 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:29 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:45210
I20260808 17:49:29.746635 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 25756
I20260808 17:49:29.762013 26340 tablet_replica.cc:330] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: stopping tablet replica
I20260808 17:49:29.762209 26340 raft_consensus.cc:2229] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:29.762426 26340 raft_consensus.cc:2258] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:29.763167 26340 ts_tablet_manager.cc:1905] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:29.764647 26340 ts_tablet_manager.cc:1918] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:29.764712 26340 log.cc:1196] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/86f549414d8841629f16ab9cdccfa329
I20260808 17:49:29.764998 26340 ts_tablet_manager.cc:1939] T 86f549414d8841629f16ab9cdccfa329 P 764a3ed9d24c4db683e449c18fef73d9: Deleting consensus metadata
I20260808 17:49:29.765465 25657 catalog_manager.cc:4841] TS 764a3ed9d24c4db683e449c18fef73d9 (127.29.62.195:37641): tablet 86f549414d8841629f16ab9cdccfa329 (table test-table [id=297cc796fa0c4720aec313d0194e1bab]) successfully deleted
I20260808 17:49:29.766913 26344 tablet_replica.cc:330] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410: stopping tablet replica
I20260808 17:49:29.767052 26344 raft_consensus.cc:2229] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:29.767198 26344 raft_consensus.cc:2258] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:29.767645 26344 ts_tablet_manager.cc:1905] T 86f549414d8841629f16ab9cdccfa329 P d7e3e5a3ddb34b68b483f6ad290d6410: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 17:49:29.767827 25657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:35889 (error 108)
I20260808 17:49:29.768522 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.768370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:35889 (local address 127.0.0.1:39744)
0808 17:49:29.768440 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.768483 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:35889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 17:49:29.768514 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 25934
W20260808 17:49:29.768649 25657 catalog_manager.cc:4584] TS 80374ad56c92453e8abb5e1a971f2ddb (127.29.62.193:35889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86f549414d8841629f16ab9cdccfa329: Network error: Client connection negotiation failed: client connection to 127.29.62.193:35889: connect: Connection refused (error 111)
I20260808 17:49:29.788851 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 26119
I20260808 17:49:29.789076 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.788882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:39653 (local address 127.0.0.1:45222)
0808 17:49:29.788985 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:29.789027 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.194:39653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 17:49:29.799803 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 25627
2026-08-08T17:49:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2140 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T17:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:29Z Disabled control of system clock
I20260808 17:49:29.837190 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40729
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:37535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:40729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:29.921732 26350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:29.921988 26350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:29.922024 26350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:29.923265 26350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:29.923328 26350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:29.923352 26350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:29.923379 26350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:29.923403 26350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:29.924787 26350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:40729
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:40729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:29.925067 26350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:29.925371 26350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:29.930169 26359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:29.930183 26358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:29.930215 26362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:29.930338 26350 server_base.cc:1034] running on GCE node
I20260808 17:49:29.930610 26350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:29.930949 26350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:29.932094 26350 hybrid_clock.cc:648] HybridClock initialized: now 1786211369932075 us; error 43 us; skew 500 ppm
I20260808 17:49:29.933388 26350 webserver.cc:460] Webserver started at http://127.29.62.254:34439/ using document root <none> and password file <none>
I20260808 17:49:29.933668 26350 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:29.933714 26350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:29.933876 26350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:29.935055 26350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "efca99d347704cb4bfb0c6d0d44f4420"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "76e23f4a3431bd5ee91f8e4a5f63aae0"
server_key_iv: "335f9f600890936df82750cfb8b5877c"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.935432 26350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "efca99d347704cb4bfb0c6d0d44f4420"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "76e23f4a3431bd5ee91f8e4a5f63aae0"
server_key_iv: "335f9f600890936df82750cfb8b5877c"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.936784 26350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:29.937515 26369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:29.937706 26350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:29.937785 26350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "efca99d347704cb4bfb0c6d0d44f4420"
format_stamp: "Formatted at 2026-08-08 17:49:29 on dist-test-slave-tk9z"
server_key: "76e23f4a3431bd5ee91f8e4a5f63aae0"
server_key_iv: "335f9f600890936df82750cfb8b5877c"
server_key_version: "encryptionkey@0"
I20260808 17:49:29.937840 26350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:29.944531 26350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:29.944767 26350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:29.944866 26350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:29.948796 26350 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:40729
I20260808 17:49:29.948827 26441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:40729 every 8 connection(s)
I20260808 17:49:29.949193 26350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:29.950204 26442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:29.952517 26442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420: Bootstrap starting.
I20260808 17:49:29.953114 26442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:29.953330 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 26350
I20260808 17:49:29.953449 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:29.953460 26442 log.cc:824] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:29.953670 29947 external_mini_cluster.cc:1428] Setting key 5cc815601e1b9774c335a460754980ca
I20260808 17:49:29.954322 26442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420: No bootstrap required, opened a new log
I20260808 17:49:29.956534 26442 raft_consensus.cc:348] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efca99d347704cb4bfb0c6d0d44f4420" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40729 } }
I20260808 17:49:29.956699 26442 raft_consensus.cc:374] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:29.956750 26442 raft_consensus.cc:729] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efca99d347704cb4bfb0c6d0d44f4420, State: Initialized, Role: FOLLOWER
I20260808 17:49:29.956890 26442 consensus_queue.cc:260] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efca99d347704cb4bfb0c6d0d44f4420" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40729 } }
I20260808 17:49:29.956975 26442 raft_consensus.cc:388] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:29.957019 26442 raft_consensus.cc:482] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:29.957067 26442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:29.957638 26442 raft_consensus.cc:504] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efca99d347704cb4bfb0c6d0d44f4420" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40729 } }
I20260808 17:49:29.957762 26442 leader_election.cc:302] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efca99d347704cb4bfb0c6d0d44f4420; no voters:
I20260808 17:49:29.958014 26442 leader_election.cc:288] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:29.958089 26452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:29.958266 26452 raft_consensus.cc:686] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [term 1 LEADER]: Becoming Leader. State: Replica: efca99d347704cb4bfb0c6d0d44f4420, State: Running, Role: LEADER
I20260808 17:49:29.958312 26442 sys_catalog.cc:564] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:29.958359 26452 consensus_queue.cc:237] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efca99d347704cb4bfb0c6d0d44f4420" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40729 } }
I20260808 17:49:29.958739 26452 sys_catalog.cc:455] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efca99d347704cb4bfb0c6d0d44f4420. Latest consensus state: current_term: 1 leader_uuid: "efca99d347704cb4bfb0c6d0d44f4420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efca99d347704cb4bfb0c6d0d44f4420" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40729 } } }
I20260808 17:49:29.958751 26454 sys_catalog.cc:455] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efca99d347704cb4bfb0c6d0d44f4420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efca99d347704cb4bfb0c6d0d44f4420" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 40729 } } }
I20260808 17:49:29.958920 26454 sys_catalog.cc:458] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:29.958834 26452 sys_catalog.cc:458] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:29.959237 26461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:29.959586 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:29.954225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.29.62.254:40729)
0808 17:49:29.954485 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:29.954494 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:29.954525 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:49:29.954940 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:29.954947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:29.955002 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:29.955148 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:29.955341 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.956214 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.957138 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:29.957553 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:29.957683 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:29.958507 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:29.958511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:29.958511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:29.958721 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:29.958765 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:29.958782 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:29.959198 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:29.959232 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 17:49:29.959368 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":81,"threads_started":1}
I20260808 17:49:29.959722 26461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:29.961810 26461 catalog_manager.cc:1347] Generated new cluster ID: 42edced1232b48adbbe6b4479d026f84
I20260808 17:49:29.961859 26461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:29.970306 26461 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:29.970825 26461 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:29.978693 26461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420: Generated new TSK 0
I20260808 17:49:29.978871 26461 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:29.983081 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40729
--builtin_ntp_servers=127.29.62.212:37535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:30.087297 26480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:30.087543 26480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:30.087584 26480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:30.088773 26480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:30.088843 26480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:30.088887 26480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:49:30.090402 26480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:30.090677 26480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:30.090986 26480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:30.096035 26487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:30.096055 26488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:30.096056 26491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:30.096277 26480 server_base.cc:1034] running on GCE node
I20260808 17:49:30.096598 26480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:30.096860 26480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:30.098032 26480 hybrid_clock.cc:648] HybridClock initialized: now 1786211370098019 us; error 28 us; skew 500 ppm
I20260808 17:49:30.099555 26480 webserver.cc:460] Webserver started at http://127.29.62.193:40491/ using document root <none> and password file <none>
I20260808 17:49:30.099818 26480 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:30.099879 26480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:30.100010 26480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:30.101537 26480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "c55beb29c8774684abc0d3c15a91fb36"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "93e129110b8be4552cbc9c6a954b8bbf"
server_key_iv: "527ee4eb58091ca6f7aa1aec62b9c39e"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.101970 26480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "c55beb29c8774684abc0d3c15a91fb36"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "93e129110b8be4552cbc9c6a954b8bbf"
server_key_iv: "527ee4eb58091ca6f7aa1aec62b9c39e"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.103629 26480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 17:49:30.104568 26497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.104780 26480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:30.104899 26480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "c55beb29c8774684abc0d3c15a91fb36"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "93e129110b8be4552cbc9c6a954b8bbf"
server_key_iv: "527ee4eb58091ca6f7aa1aec62b9c39e"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.104995 26480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:30.137207 26480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:30.137513 26480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:30.137624 26480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:30.137838 26480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:30.138206 26480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:30.138242 26480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.138305 26480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:30.138330 26480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.144479 26480 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:45157
I20260808 17:49:30.144526 26650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:45157 every 8 connection(s)
I20260808 17:49:30.144883 26480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:49:30.149919 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.145613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:53079 (local address 127.29.62.254:40729)
0808 17:49:30.145692 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.145695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.145962 (+ 267us) server_negotiation.cc:407] Connection header received
0808 17:49:30.146515 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.146515 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.146525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.146588 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:30.147268 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.147765 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.148697 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.148873 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.148923 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.149691 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:30.149692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:30.149693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:30.149697 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:30.149710 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:30.149713 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:30.149756 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.149838 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.149859 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:49:30.150130 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.145615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40729 (local address 127.29.62.193:53079)
0808 17:49:30.145817 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.145862 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.146418 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.146663 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.146667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.146675 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:30.147188 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.147193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.147848 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.147849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.148625 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.148628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.148956 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.148957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.149070 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.149562 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:30.149562 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:30.149591 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:30.149614 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:30.149767 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:30.149771 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:30.149774 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.149826 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.149914 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260808 17:49:30.150221 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 26480
I20260808 17:49:30.150341 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:49:30.150578 29947 external_mini_cluster.cc:1428] Setting key b9cb033b21a1ce7f0696b640bf61a195
I20260808 17:49:30.150841 26651 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40729
I20260808 17:49:30.150950 26651 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:30.151188 26651 heartbeater.cc:511] Master 127.29.62.254:40729 requested a full tablet report, sending...
I20260808 17:49:30.151656 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40729
--builtin_ntp_servers=127.29.62.212:37535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:49:30.151763 26391 ts_manager.cc:194] Registered new tserver with Master: c55beb29c8774684abc0d3c15a91fb36 (127.29.62.193:45157)
I20260808 17:49:30.152645 26391 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.193:53079
W20260808 17:49:30.233707 26655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:30.233901 26655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:30.233922 26655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:30.235103 26655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:30.235144 26655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:30.235168 26655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:49:30.236511 26655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:30.236716 26655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:30.237010 26655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:30.241983 26662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:30.241997 26663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:30.241999 26665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:30.242292 26655 server_base.cc:1034] running on GCE node
I20260808 17:49:30.242555 26655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:30.242792 26655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:30.243932 26655 hybrid_clock.cc:648] HybridClock initialized: now 1786211370243918 us; error 42 us; skew 500 ppm
I20260808 17:49:30.245242 26655 webserver.cc:460] Webserver started at http://127.29.62.194:41515/ using document root <none> and password file <none>
I20260808 17:49:30.245487 26655 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:30.245535 26655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:30.245697 26655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:30.246822 26655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "8ef343cda3584d14b8332a927144a246"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "31085feddd98fab147badbccaed93c0c"
server_key_iv: "88d7439453a3ebeff0297262dba684c9"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.247201 26655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "8ef343cda3584d14b8332a927144a246"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "31085feddd98fab147badbccaed93c0c"
server_key_iv: "88d7439453a3ebeff0297262dba684c9"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.248512 26655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:30.249217 26675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.249424 26655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:30.249538 26655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "8ef343cda3584d14b8332a927144a246"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "31085feddd98fab147badbccaed93c0c"
server_key_iv: "88d7439453a3ebeff0297262dba684c9"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.249619 26655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:30.264807 26655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:30.265079 26655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:30.265228 26655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:30.265451 26655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:30.265779 26655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:30.265834 26655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.265869 26655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:30.265897 26655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.271631 26655 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:40207
I20260808 17:49:30.271679 26822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:40207 every 8 connection(s)
I20260808 17:49:30.271978 26655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:49:30.277175 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.272659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:55953 (local address 127.29.62.254:40729)
0808 17:49:30.272728 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.272731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.272969 (+ 238us) server_negotiation.cc:407] Connection header received
0808 17:49:30.273460 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.273462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.273474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.273538 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:30.274186 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.274847 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.275826 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.276037 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.276103 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.276932 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:30.276934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:30.276934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:30.276939 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:30.276956 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:30.276960 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:30.277014 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.277087 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.277110 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:49:30.277329 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.272659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40729 (local address 127.29.62.194:55953)
0808 17:49:30.272852 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.272884 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.273333 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.273586 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.273589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.273597 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:30.274079 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.274082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.274931 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.274932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.275717 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.275719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.276121 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.276122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.276227 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.276785 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:30.276786 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:30.276814 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:30.276837 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:30.277017 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:30.277019 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:30.277021 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.277068 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.277138 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260808 17:49:30.277971 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 26655
I20260808 17:49:30.278029 26823 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40729
I20260808 17:49:30.278085 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:49:30.278143 26823 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:30.278297 29947 external_mini_cluster.cc:1428] Setting key 1b2275c7f7b2d09b6d90f1e684f31626
I20260808 17:49:30.278323 26823 heartbeater.cc:511] Master 127.29.62.254:40729 requested a full tablet report, sending...
I20260808 17:49:30.278791 26390 ts_manager.cc:194] Registered new tserver with Master: 8ef343cda3584d14b8332a927144a246 (127.29.62.194:40207)
I20260808 17:49:30.279210 26390 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.194:55953
I20260808 17:49:30.279292 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40729
--builtin_ntp_servers=127.29.62.212:37535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:30.362226 26827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:30.362493 26827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:30.362591 26827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:30.363734 26827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:30.363772 26827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:30.363819 26827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:49:30.365231 26827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:37535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:40729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:30.365442 26827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:30.365722 26827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:30.370648 26833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:30.370718 26827 server_base.cc:1034] running on GCE node
W20260808 17:49:30.370667 26835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:30.370667 26832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:30.371058 26827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:30.371312 26827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:30.372483 26827 hybrid_clock.cc:648] HybridClock initialized: now 1786211370372471 us; error 28 us; skew 500 ppm
I20260808 17:49:30.373728 26827 webserver.cc:460] Webserver started at http://127.29.62.195:43965/ using document root <none> and password file <none>
I20260808 17:49:30.373997 26827 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:30.374069 26827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:30.374221 26827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:30.375437 26827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "4f5c93d3f4024409ac25506ed509ae1b"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "759b9c36c514eabc0f434d124144354d"
server_key_iv: "09c9f2b3be0daf4c159021949f73906c"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.375823 26827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "4f5c93d3f4024409ac25506ed509ae1b"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "759b9c36c514eabc0f434d124144354d"
server_key_iv: "09c9f2b3be0daf4c159021949f73906c"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.377139 26827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:30.377833 26841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.378023 26827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:30.378129 26827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "4f5c93d3f4024409ac25506ed509ae1b"
format_stamp: "Formatted at 2026-08-08 17:49:30 on dist-test-slave-tk9z"
server_key: "759b9c36c514eabc0f434d124144354d"
server_key_iv: "09c9f2b3be0daf4c159021949f73906c"
server_key_version: "encryptionkey@0"
I20260808 17:49:30.378204 26827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:30.387434 26827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:30.387753 26827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:30.387910 26827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:30.388151 26827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:30.388509 26827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:30.388568 26827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.388614 26827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:30.388639 26827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:30.394492 26827 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:39353
I20260808 17:49:30.394537 26989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:39353 every 8 connection(s)
I20260808 17:49:30.394944 26827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:49:30.399828 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.395659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:44271 (local address 127.29.62.254:40729)
0808 17:49:30.395705 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.395707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.395941 (+ 234us) server_negotiation.cc:407] Connection header received
0808 17:49:30.396414 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.396415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.396427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.396481 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:30.397115 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.397603 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.398617 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.398804 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.398857 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.399602 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:30.399603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:30.399604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:30.399608 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:30.399622 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:30.399626 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:30.399673 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.399739 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.399764 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:49:30.400014 26993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.395642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:40729 (local address 127.29.62.195:44271)
0808 17:49:30.395808 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.395842 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.396329 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.396534 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.396537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.396546 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:30.397047 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.397051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.397684 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.397685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.398542 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.398545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.398877 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.398878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.398991 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.399469 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:30.399470 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:30.399497 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:30.399520 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:30.399675 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:30.399678 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:30.399680 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.399726 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.399802 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260808 17:49:30.400519 26990 heartbeater.cc:347] Connected to a master server at 127.29.62.254:40729
I20260808 17:49:30.400635 26990 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:30.400825 26990 heartbeater.cc:511] Master 127.29.62.254:40729 requested a full tablet report, sending...
I20260808 17:49:30.401237 26390 ts_manager.cc:194] Registered new tserver with Master: 4f5c93d3f4024409ac25506ed509ae1b (127.29.62.195:39353)
I20260808 17:49:30.401624 26390 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.195:44271
I20260808 17:49:30.405100 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 26827
I20260808 17:49:30.405190 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:49:30.405395 29947 external_mini_cluster.cc:1428] Setting key 5fb1b61cef3ec096256967386b6e1f67
I20260808 17:49:30.406320 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:49:30.410741 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.407800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34494 (local address 127.29.62.254:40729)
0808 17:49:30.407865 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.407866 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.407967 (+ 101us) server_negotiation.cc:407] Connection header received
0808 17:49:30.408045 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.408046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.408055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.408106 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:30.408275 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.408754 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.409570 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.409751 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.409801 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.410535 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:30.410536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:30.410537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:30.410540 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:30.410553 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:30.410557 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:30.410630 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.410673 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.410683 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:49:30.412436 26390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:49:30.414024 26390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:49:30.420657 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.416677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.29.62.195:39353)
0808 17:49:30.416993 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.416997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.417025 (+ 28us) server_negotiation.cc:407] Connection header received
0808 17:49:30.417066 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.417068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.417084 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.417164 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:30.417378 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.418542 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.419755 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.420243 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.420329 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.420423 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.420516 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.420554 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":111,"threads_started":1}
I20260808 17:49:30.420701 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.416351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:45157 (local address 127.0.0.1:41854)
0808 17:49:30.416669 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.416679 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.416735 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.417104 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.417107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.417110 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:30.417183 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.417186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.418177 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.418178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.419498 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.419501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.420421 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.420422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.420484 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.420489 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.420522 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.420573 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":25,"spinlock_wait_cycles":768,"thread_start_us":117,"threads_started":1}
I20260808 17:49:30.420701 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.416362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:39353 (local address 127.0.0.1:34980)
0808 17:49:30.416829 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.416838 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.416872 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.417242 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.417243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.417245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:30.417296 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.417298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.418633 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.418634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.419672 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.419673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.420343 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.420344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.420418 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.420434 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.420503 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.420556 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":1,"thread_start_us":139,"threads_started":1}
I20260808 17:49:30.420873 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.416350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:40207 (local address 127.0.0.1:49118)
0808 17:49:30.416525 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.416567 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.416665 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.417332 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.417333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.417335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:30.417370 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.417371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.418724 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.418725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.419706 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.419709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.420614 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.420615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.420677 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.420680 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.420710 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.420750 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":4,"thread_start_us":106,"threads_started":1}
I20260808 17:49:30.421118 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.416671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.29.62.193:45157)
0808 17:49:30.416878 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.416883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.416897 (+ 14us) server_negotiation.cc:407] Connection header received
0808 17:49:30.416940 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.416942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.416960 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.417269 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:30.417342 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.418082 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.419592 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.420303 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.420814 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.420931 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.420963 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.421017 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":132,"threads_started":1}
I20260808 17:49:30.421512 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.416920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.29.62.194:40207)
0808 17:49:30.417165 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.417169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.417179 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:49:30.417217 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.417219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.417234 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.417459 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:30.417523 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.418606 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.419804 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.420522 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.421147 (+ 625us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.421340 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.421371 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.421417 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260808 17:49:30.421592 26904 tablet_service.cc:1467] Processing CreateTablet for tablet 76f57b11a7464baba88878534a7a8855 (DEFAULT_TABLE table=test-table [id=13aba8112f644d7aa9480a1708e9e8f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:30.422037 26904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f57b11a7464baba88878534a7a8855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:30.422008 26562 tablet_service.cc:1467] Processing CreateTablet for tablet 76f57b11a7464baba88878534a7a8855 (DEFAULT_TABLE table=test-table [id=13aba8112f644d7aa9480a1708e9e8f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:30.422410 26562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f57b11a7464baba88878534a7a8855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:30.422406 26731 tablet_service.cc:1467] Processing CreateTablet for tablet 76f57b11a7464baba88878534a7a8855 (DEFAULT_TABLE table=test-table [id=13aba8112f644d7aa9480a1708e9e8f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:30.422747 26731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f57b11a7464baba88878534a7a8855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:30.425434 27011 tablet_bootstrap.cc:492] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: Bootstrap starting.
I20260808 17:49:30.425846 27012 tablet_bootstrap.cc:492] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: Bootstrap starting.
I20260808 17:49:30.425902 27014 tablet_bootstrap.cc:492] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36: Bootstrap starting.
I20260808 17:49:30.426278 27011 tablet_bootstrap.cc:654] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:30.426533 27014 tablet_bootstrap.cc:654] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:30.426568 27012 tablet_bootstrap.cc:654] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:30.426640 27011 log.cc:824] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:30.426855 27014 log.cc:824] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:30.426888 27012 log.cc:824] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:30.427464 27011 tablet_bootstrap.cc:492] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: No bootstrap required, opened a new log
I20260808 17:49:30.427554 27011 ts_tablet_manager.cc:1397] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:49:30.427654 27012 tablet_bootstrap.cc:492] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: No bootstrap required, opened a new log
I20260808 17:49:30.427698 27014 tablet_bootstrap.cc:492] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36: No bootstrap required, opened a new log
I20260808 17:49:30.427724 27012 ts_tablet_manager.cc:1397] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:49:30.427753 27014 ts_tablet_manager.cc:1397] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 17:49:30.429561 27012 raft_consensus.cc:348] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.429723 27012 raft_consensus.cc:374] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:30.429766 27012 raft_consensus.cc:729] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f5c93d3f4024409ac25506ed509ae1b, State: Initialized, Role: FOLLOWER
I20260808 17:49:30.429874 27012 consensus_queue.cc:260] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.430178 27012 ts_tablet_manager.cc:1428] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:49:30.430200 26990 heartbeater.cc:503] Master 127.29.62.254:40729 was elected leader, sending a full tablet report...
I20260808 17:49:30.430215 27011 raft_consensus.cc:348] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.430300 27014 raft_consensus.cc:348] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.430421 27011 raft_consensus.cc:374] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:30.430456 27014 raft_consensus.cc:374] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:30.430459 27011 raft_consensus.cc:729] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef343cda3584d14b8332a927144a246, State: Initialized, Role: FOLLOWER
I20260808 17:49:30.430491 27014 raft_consensus.cc:729] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55beb29c8774684abc0d3c15a91fb36, State: Initialized, Role: FOLLOWER
I20260808 17:49:30.430601 27011 consensus_queue.cc:260] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.430608 27014 consensus_queue.cc:260] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.430857 27014 ts_tablet_manager.cc:1428] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 17:49:30.430857 27011 ts_tablet_manager.cc:1428] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 17:49:30.430917 26651 heartbeater.cc:503] Master 127.29.62.254:40729 was elected leader, sending a full tablet report...
I20260808 17:49:30.430997 26823 heartbeater.cc:503] Master 127.29.62.254:40729 was elected leader, sending a full tablet report...
I20260808 17:49:30.481174 27020 raft_consensus.cc:482] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:30.481339 27020 raft_consensus.cc:504] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.481478 27021 raft_consensus.cc:482] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:30.481698 27021 raft_consensus.cc:504] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.482086 27020 leader_election.cc:288] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f5c93d3f4024409ac25506ed509ae1b (127.29.62.195:39353), 8ef343cda3584d14b8332a927144a246 (127.29.62.194:40207)
I20260808 17:49:30.482690 27021 leader_election.cc:288] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f5c93d3f4024409ac25506ed509ae1b (127.29.62.195:39353), c55beb29c8774684abc0d3c15a91fb36 (127.29.62.193:45157)
I20260808 17:49:30.485692 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.481993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:39353 (local address 127.29.62.193:53643)
0808 17:49:30.482052 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.482066 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.482112 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.482275 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.482276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.482278 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:30.482356 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.482359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.483438 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.483439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.484665 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.484668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.485404 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.485413 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.485496 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.485501 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.485551 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.485604 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 17:49:30.485726 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.482414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:33933 (local address 127.29.62.194:40207)
0808 17:49:30.482465 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.482467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.482556 (+ 89us) server_negotiation.cc:407] Connection header received
0808 17:49:30.482645 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.482646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.482666 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.482696 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:30.482922 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.483581 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.484838 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.485346 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.485420 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.485427 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.485628 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.485664 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:49:30.485894 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.482276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:39353 (local address 127.29.62.194:57355)
0808 17:49:30.482326 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.482337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.482398 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.482911 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.482912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.482914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:30.483022 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.483024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.484001 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.484001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.484899 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.484902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.485670 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.485671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.485741 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.485744 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.485776 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.485826 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":16}
I20260808 17:49:30.486002 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.482061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:53643 (local address 127.29.62.195:39353)
0808 17:49:30.482107 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.482110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.482122 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:49:30.482203 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.482204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.482216 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.482271 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:30.482500 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.483265 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.484762 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.485297 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.485362 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.485371 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.485868 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.485908 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:49:30.486304 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.482276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:40207 (local address 127.29.62.193:33933)
0808 17:49:30.482498 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.482509 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.482577 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.482785 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.482786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.482788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:30.482852 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.482854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.483668 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.483669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.484775 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.484777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.485474 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.485476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.485549 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.485553 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.486179 (+ 626us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.486226 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":131,"threads_started":1}
I20260808 17:49:30.486264 26933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f57b11a7464baba88878534a7a8855" candidate_uuid: "c55beb29c8774684abc0d3c15a91fb36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5c93d3f4024409ac25506ed509ae1b" is_pre_election: true
I20260808 17:49:30.486446 26933 raft_consensus.cc:2454] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c55beb29c8774684abc0d3c15a91fb36 in term 0.
I20260808 17:49:30.486677 27025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.482522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:57355 (local address 127.29.62.195:39353)
0808 17:49:30.482763 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.482765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.482774 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:49:30.482802 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.482802 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.482813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.483088 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:30.483127 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.483884 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.484995 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.485571 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.486064 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.486074 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.486095 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.486135 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":80,"threads_started":1}
I20260808 17:49:30.486634 26763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f57b11a7464baba88878534a7a8855" candidate_uuid: "c55beb29c8774684abc0d3c15a91fb36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef343cda3584d14b8332a927144a246" is_pre_election: true
I20260808 17:49:30.486738 26763 raft_consensus.cc:2454] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c55beb29c8774684abc0d3c15a91fb36 in term 0.
I20260808 17:49:30.486727 26502 leader_election.cc:302] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5c93d3f4024409ac25506ed509ae1b, c55beb29c8774684abc0d3c15a91fb36; no voters:
I20260808 17:49:30.486827 26933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f57b11a7464baba88878534a7a8855" candidate_uuid: "8ef343cda3584d14b8332a927144a246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5c93d3f4024409ac25506ed509ae1b" is_pre_election: true
I20260808 17:49:30.486858 27020 raft_consensus.cc:2781] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:49:30.486897 26933 raft_consensus.cc:2454] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef343cda3584d14b8332a927144a246 in term 0.
I20260808 17:49:30.486918 27020 raft_consensus.cc:482] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:49:30.486943 27020 raft_consensus.cc:3037] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:30.487092 26679 leader_election.cc:302] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f5c93d3f4024409ac25506ed509ae1b, 8ef343cda3584d14b8332a927144a246; no voters:
I20260808 17:49:30.487227 27021 raft_consensus.cc:2781] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:49:30.487290 27021 raft_consensus.cc:482] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:49:30.487324 27021 raft_consensus.cc:3037] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:30.487793 27020 raft_consensus.cc:504] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.487944 27020 leader_election.cc:288] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [CANDIDATE]: Term 1 election: Requested vote from peers 4f5c93d3f4024409ac25506ed509ae1b (127.29.62.195:39353), 8ef343cda3584d14b8332a927144a246 (127.29.62.194:40207)
I20260808 17:49:30.488060 27021 raft_consensus.cc:504] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.488112 26933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f57b11a7464baba88878534a7a8855" candidate_uuid: "c55beb29c8774684abc0d3c15a91fb36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5c93d3f4024409ac25506ed509ae1b"
I20260808 17:49:30.488137 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.482695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:41129 (local address 127.29.62.193:45157)
0808 17:49:30.482743 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.482744 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.482863 (+ 119us) server_negotiation.cc:407] Connection header received
0808 17:49:30.482963 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.482964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.482975 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.483006 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:30.483212 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.483911 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.485997 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.486557 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.486610 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.486618 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.488013 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.488059 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:49:30.488173 26933 raft_consensus.cc:3037] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:30.488211 27021 leader_election.cc:288] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [CANDIDATE]: Term 1 election: Requested vote from peers 4f5c93d3f4024409ac25506ed509ae1b (127.29.62.195:39353), c55beb29c8774684abc0d3c15a91fb36 (127.29.62.193:45157)
I20260808 17:49:30.488349 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.482599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:45157 (local address 127.29.62.194:41129)
0808 17:49:30.482802 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.482811 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:30.482889 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:30.483066 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:30.483066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:30.483068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:30.483129 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.483130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.484000 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.484001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.485887 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:30.485890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.486679 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.486680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:30.486749 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.486753 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:49:30.488203 (+ 1450us) client_negotiation.cc:240] Negotiation successful
0808 17:49:30.488257 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":33,"thread_start_us":62,"threads_started":1}
I20260808 17:49:30.488220 26763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f57b11a7464baba88878534a7a8855" candidate_uuid: "c55beb29c8774684abc0d3c15a91fb36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef343cda3584d14b8332a927144a246"
I20260808 17:49:30.488420 26929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f57b11a7464baba88878534a7a8855" candidate_uuid: "8ef343cda3584d14b8332a927144a246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f5c93d3f4024409ac25506ed509ae1b"
I20260808 17:49:30.488498 26763 raft_consensus.cc:2379] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c55beb29c8774684abc0d3c15a91fb36 in current term 1: Already voted for candidate 8ef343cda3584d14b8332a927144a246 in this term.
I20260808 17:49:30.488849 26588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f57b11a7464baba88878534a7a8855" candidate_uuid: "8ef343cda3584d14b8332a927144a246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55beb29c8774684abc0d3c15a91fb36"
I20260808 17:49:30.488845 26590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f57b11a7464baba88878534a7a8855" candidate_uuid: "8ef343cda3584d14b8332a927144a246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55beb29c8774684abc0d3c15a91fb36" is_pre_election: true
I20260808 17:49:30.489007 26588 raft_consensus.cc:2379] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8ef343cda3584d14b8332a927144a246 in current term 1: Already voted for candidate c55beb29c8774684abc0d3c15a91fb36 in this term.
I20260808 17:49:30.489117 26933 raft_consensus.cc:2454] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c55beb29c8774684abc0d3c15a91fb36 in term 1.
I20260808 17:49:30.489270 26680 leader_election.cc:302] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ef343cda3584d14b8332a927144a246; no voters: 4f5c93d3f4024409ac25506ed509ae1b, c55beb29c8774684abc0d3c15a91fb36
I20260808 17:49:30.489388 26502 leader_election.cc:302] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f5c93d3f4024409ac25506ed509ae1b, c55beb29c8774684abc0d3c15a91fb36; no voters: 8ef343cda3584d14b8332a927144a246
I20260808 17:49:30.489431 27021 raft_consensus.cc:2726] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 17:49:30.489514 27020 raft_consensus.cc:2781] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:30.489656 27020 raft_consensus.cc:686] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 1 LEADER]: Becoming Leader. State: Replica: c55beb29c8774684abc0d3c15a91fb36, State: Running, Role: LEADER
I20260808 17:49:30.489748 27020 consensus_queue.cc:237] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } }
I20260808 17:49:30.490612 26388 catalog_manager.cc:5495] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 reported cstate change: term changed from 0 to 1, leader changed from <none> to c55beb29c8774684abc0d3c15a91fb36 (127.29.62.193). New cstate: current_term: 1 leader_uuid: "c55beb29c8774684abc0d3c15a91fb36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c55beb29c8774684abc0d3c15a91fb36" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 } health_report { overall_health: UNKNOWN } } }
I20260808 17:49:30.512359 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.510149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.29.62.193:45157)
0808 17:49:30.510207 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:30.510209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:30.510220 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:49:30.510269 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:30.510270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:30.510286 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:30.510350 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:49:30.510543 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.511006 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.511718 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:30.511894 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:30.511945 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:30.512073 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:49:30.512157 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:49:30.512225 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:30.512274 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:49:30.512283 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 17:49:30.514175 26763 raft_consensus.cc:1261] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 1 FOLLOWER]: Refusing update from remote peer c55beb29c8774684abc0d3c15a91fb36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:30.514189 26933 raft_consensus.cc:1261] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 1 FOLLOWER]: Refusing update from remote peer c55beb29c8774684abc0d3c15a91fb36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:30.514477 27028 consensus_queue.cc:1035] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef343cda3584d14b8332a927144a246" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 40207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:30.514602 27020 consensus_queue.cc:1035] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f5c93d3f4024409ac25506ed509ae1b" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 39353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:30.517153 27032 mvcc.cc:205] Tried to move back new op lower bound from 7316321773623422976 to 7316321773526605824. Current Snapshot: MvccSnapshot[applied={T|T < 7316321773623422976}]
I20260808 17:49:30.517486 27033 mvcc.cc:205] Tried to move back new op lower bound from 7316321773623422976 to 7316321773526605824. Current Snapshot: MvccSnapshot[applied={T|T < 7316321773623422976}]
I20260808 17:49:30.518102 27034 mvcc.cc:205] Tried to move back new op lower bound from 7316321773623422976 to 7316321773526605824. Current Snapshot: MvccSnapshot[applied={T|T < 7316321773623422976}]
I20260808 17:49:30.519491 26390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:30.519598 26390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:30.520357 26390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efca99d347704cb4bfb0c6d0d44f4420: Sending DeleteTablet for 3 replicas of tablet 76f57b11a7464baba88878534a7a8855
I20260808 17:49:30.520728 26904 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f57b11a7464baba88878534a7a8855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:30 UTC) from {username='slave'} at 127.0.0.1:34980
I20260808 17:49:30.520754 26562 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f57b11a7464baba88878534a7a8855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:30 UTC) from {username='slave'} at 127.0.0.1:41854
I20260808 17:49:30.520982 27054 tablet_replica.cc:330] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36: stopping tablet replica
I20260808 17:49:30.521077 27054 raft_consensus.cc:2229] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:49:30.521102 26731 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f57b11a7464baba88878534a7a8855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:30 UTC) from {username='slave'} at 127.0.0.1:49118
I20260808 17:49:30.521162 27054 raft_consensus.cc:2258] T 76f57b11a7464baba88878534a7a8855 P c55beb29c8774684abc0d3c15a91fb36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:30.521234 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 26480
I20260808 17:49:30.521303 27053 tablet_replica.cc:330] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: stopping tablet replica
I20260808 17:49:30.521384 27053 raft_consensus.cc:2229] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:30.521467 27053 raft_consensus.cc:2258] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:30.525345 27053 ts_tablet_manager.cc:1905] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:30.527097 27053 ts_tablet_manager.cc:1918] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:30.527160 27053 log.cc:1196] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/76f57b11a7464baba88878534a7a8855
I20260808 17:49:30.527372 27053 ts_tablet_manager.cc:1939] T 76f57b11a7464baba88878534a7a8855 P 4f5c93d3f4024409ac25506ed509ae1b: Deleting consensus metadata
I20260808 17:49:30.532765 26373 catalog_manager.cc:4841] TS 4f5c93d3f4024409ac25506ed509ae1b (127.29.62.195:39353): tablet 76f57b11a7464baba88878534a7a8855 (table test-table [id=13aba8112f644d7aa9480a1708e9e8f0]) successfully deleted
I20260808 17:49:30.532961 27055 tablet_replica.cc:330] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: stopping tablet replica
I20260808 17:49:30.533105 27055 raft_consensus.cc:2229] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:30.533221 27055 raft_consensus.cc:2258] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:30.533671 27055 ts_tablet_manager.cc:1905] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:30.535163 27055 ts_tablet_manager.cc:1918] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:30.535255 27055 log.cc:1196] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/76f57b11a7464baba88878534a7a8855
I20260808 17:49:30.535492 27055 ts_tablet_manager.cc:1939] T 76f57b11a7464baba88878534a7a8855 P 8ef343cda3584d14b8332a927144a246: Deleting consensus metadata
I20260808 17:49:30.535894 26370 catalog_manager.cc:4841] TS 8ef343cda3584d14b8332a927144a246 (127.29.62.194:40207): tablet 76f57b11a7464baba88878534a7a8855 (table test-table [id=13aba8112f644d7aa9480a1708e9e8f0]) successfully deleted
W20260808 17:49:30.536794 26375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:45157 (error 108)
I20260808 17:49:30.537017 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 26655
I20260808 17:49:30.538827 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:30.537136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:45157 (local address 127.0.0.1:41872)
0808 17:49:30.538724 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:30.538781 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:45157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1563}
W20260808 17:49:30.542312 26375 catalog_manager.cc:4584] TS c55beb29c8774684abc0d3c15a91fb36 (127.29.62.193:45157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 76f57b11a7464baba88878534a7a8855: Network error: Client connection negotiation failed: client connection to 127.29.62.193:45157: connect: Connection refused (error 111)
I20260808 17:49:30.552342 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 26827
I20260808 17:49:30.565866 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 26350
2026-08-08T17:49:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (764 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:30 dist-test-slave-tk9z krb5kdc[27059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:30 dist-test-slave-tk9z krb5kdc[27059](info): set up 2 sockets
Aug 08 17:49:30 dist-test-slave-tk9z krb5kdc[27059](info): commencing operation
krb5kdc: starting...
W20260808 17:49:31.876214 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:31 dist-test-slave-tk9z krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:49:31.997062 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:41707
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:41707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:32.087004 27076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:32.087217 27076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:32.087240 27076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:32.088359 27076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:32.088402 27076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:32.088415 27076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:32.088428 27076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:32.088440 27076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:32.089949 27076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:41707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:41707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:32.090165 27076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:32.090440 27076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:32.095391 27082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:32.095414 27083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:32.095393 27089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:32.095682 27076 server_base.cc:1034] running on GCE node
I20260808 17:49:32.095995 27076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:32.096318 27076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:32.097503 27076 hybrid_clock.cc:648] HybridClock initialized: now 1786211372097495 us; error 37 us; skew 500 ppm
Aug 08 17:49:32 dist-test-slave-tk9z krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211372, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:32.100054 27076 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:32.100492 27076 webserver.cc:460] Webserver started at http://127.29.62.254:35643/ using document root <none> and password file <none>
I20260808 17:49:32.100739 27076 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:32.100807 27076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:32.100957 27076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:32.102038 27076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "72df34ee7d994f89add0733ad78953a6"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "31d7b74c253f32625559af131679d63c"
server_key_iv: "252e6cb2b38ffb35a892082afdaa12e7"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.102478 27076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "72df34ee7d994f89add0733ad78953a6"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "31d7b74c253f32625559af131679d63c"
server_key_iv: "252e6cb2b38ffb35a892082afdaa12e7"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.103832 27076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:32.104600 27097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.104779 27076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:49:32.104897 27076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "72df34ee7d994f89add0733ad78953a6"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "31d7b74c253f32625559af131679d63c"
server_key_iv: "252e6cb2b38ffb35a892082afdaa12e7"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.104974 27076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:32.122562 27076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:32.124593 27076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:32.124766 27076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:32.128878 27076 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:41707
I20260808 17:49:32.128930 27168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:41707 every 8 connection(s)
I20260808 17:49:32.129323 27076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:32.130296 27170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:32.132824 27170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6: Bootstrap starting.
I20260808 17:49:32.133203 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 27076
I20260808 17:49:32.133353 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:32.133571 29947 external_mini_cluster.cc:1428] Setting key 1bfd9d660f1518487f7385393c53fc16
I20260808 17:49:32.133631 27170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:32.133966 27170 log.cc:824] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:32.134806 27170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6: No bootstrap required, opened a new log
I20260808 17:49:32.136709 27170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72df34ee7d994f89add0733ad78953a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41707 } }
I20260808 17:49:32.136883 27170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:32.136933 27170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72df34ee7d994f89add0733ad78953a6, State: Initialized, Role: FOLLOWER
I20260808 17:49:32.137075 27170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72df34ee7d994f89add0733ad78953a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41707 } }
I20260808 17:49:32.137162 27170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:32.137208 27170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:32.137257 27170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:32.137822 27170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72df34ee7d994f89add0733ad78953a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41707 } }
I20260808 17:49:32.137948 27170 leader_election.cc:302] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72df34ee7d994f89add0733ad78953a6; no voters:
I20260808 17:49:32.138145 27170 leader_election.cc:288] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:32.138319 27178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:32.138456 27170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:32.138551 27178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [term 1 LEADER]: Becoming Leader. State: Replica: 72df34ee7d994f89add0733ad78953a6, State: Running, Role: LEADER
I20260808 17:49:32.138653 27178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72df34ee7d994f89add0733ad78953a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41707 } }
I20260808 17:49:32.138983 27177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72df34ee7d994f89add0733ad78953a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72df34ee7d994f89add0733ad78953a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41707 } } }
I20260808 17:49:32.139071 27177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:32.139319 27187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:32.139846 27187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:49:32 dist-test-slave-tk9z krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:32.141077 27179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72df34ee7d994f89add0733ad78953a6. Latest consensus state: current_term: 1 leader_uuid: "72df34ee7d994f89add0733ad78953a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72df34ee7d994f89add0733ad78953a6" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41707 } } }
I20260808 17:49:32.141181 27179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:32.142413 27187 catalog_manager.cc:1347] Generated new cluster ID: a1c4fff647324cf0b3e720f77dffd716
I20260808 17:49:32.143239 27187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:32.144840 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.134142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.29.62.254:41707)
0808 17:49:32.134327 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.134336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.134392 (+ 56us) server_negotiation.cc:407] Connection header received
0808 17:49:32.134919 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.134927 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.134985 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.135130 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:32.135778 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.136675 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.137594 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.138089 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.138237 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.141263 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:32.141267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:32.141268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:32.141282 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:32.143168 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.143595 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.143598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.143599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.143702 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.143944 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.143945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.143945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.144232 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:32.144432 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.144507 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.144637 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260808 17:49:32.150038 27187 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:32.150621 27187 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:32.159969 27187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6: Generated new TSK 0
I20260808 17:49:32.160182 27187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.62.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
Entry for principal kudu/127.29.62.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab.
I20260808 17:49:32.189347 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:41707
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:32.274606 27205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:32.274814 27205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:32.274844 27205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:32.276082 27205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:32.276127 27205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:32.276181 27205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:49:32.277681 27205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.193.keytab
--principal=kudu/127.29.62.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:41707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:32.277961 27205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:32.278213 27205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:32.283672 27213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:32.283702 27218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:32.283695 27214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:32.283878 27205 server_base.cc:1034] running on GCE node
I20260808 17:49:32.284099 27205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:32.284319 27205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:32.285463 27205 hybrid_clock.cc:648] HybridClock initialized: now 1786211372285447 us; error 40 us; skew 500 ppm
Aug 08 17:49:32 dist-test-slave-tk9z krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211372, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:32.288105 27205 init.cc:377] Logged in from keytab as kudu/127.29.62.193@KRBTEST.COM (short username kudu)
I20260808 17:49:32.288563 27205 webserver.cc:460] Webserver started at http://127.29.62.193:46835/ using document root <none> and password file <none>
I20260808 17:49:32.288834 27205 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:32.288906 27205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:32.289065 27205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:32.290158 27205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "bd2ef38ff69b4e82bb5838016e975fa1"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "cac1dabb9f6bf13d4bfbb9f875188da7"
server_key_iv: "7ce606c21f462bfb983b61606424d0e4"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.290587 27205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "bd2ef38ff69b4e82bb5838016e975fa1"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "cac1dabb9f6bf13d4bfbb9f875188da7"
server_key_iv: "7ce606c21f462bfb983b61606424d0e4"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.292043 27205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:32.292791 27226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.292968 27205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:49:32.293078 27205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "bd2ef38ff69b4e82bb5838016e975fa1"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "cac1dabb9f6bf13d4bfbb9f875188da7"
server_key_iv: "7ce606c21f462bfb983b61606424d0e4"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.293155 27205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:32.314790 27205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:32.317250 27205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:32.317431 27205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:32.317683 27205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:32.318020 27205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:32.318076 27205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.318117 27205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:32.318164 27205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.324074 27205 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:32875
I20260808 17:49:32.324128 27380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:32875 every 8 connection(s)
I20260808 17:49:32.324460 27205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:49:32.326151 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 27205
I20260808 17:49:32.326289 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:49:32.326565 29947 external_mini_cluster.cc:1428] Setting key e0ebf091b541db1761d193d25f32a78d
Aug 08 17:49:32 dist-test-slave-tk9z krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211372, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.193@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:32.334455 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.325214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:40147 (local address 127.29.62.254:41707)
0808 17:49:32.325279 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.325281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.325479 (+ 198us) server_negotiation.cc:407] Connection header received
0808 17:49:32.325986 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.325987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.325996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.326054 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:32.327234 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.327751 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.328772 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.328994 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.329071 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.331473 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:32.331475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:32.331475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:32.331485 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:32.332925 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.333423 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.333423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.333424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.333476 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.333754 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.333754 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.333755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.333857 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:32.333907 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.334254 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.334330 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:49:32.334735 27385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.325185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:41707 (local address 127.29.62.193:40147)
0808 17:49:32.325349 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.325392 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.325880 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.326137 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.326142 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.326933 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:32.327128 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.327133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.327876 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.327877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.328668 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.328670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.329081 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.329082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.329207 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.329737 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:32.329746 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:32.331363 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:32.333012 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:32.333021 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:32.333023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:32.333336 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:32.333542 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:32.333543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:32.333543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:32.333687 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:32.333930 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:32.333937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:32.334108 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.334238 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.334443 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260808 17:49:32.335268 27381 heartbeater.cc:347] Connected to a master server at 127.29.62.254:41707
I20260808 17:49:32.335371 27381 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:32.335582 27381 heartbeater.cc:511] Master 127.29.62.254:41707 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.62.194@KRBTEST.COM; defaulting to no policy
I20260808 17:49:32.336145 27119 ts_manager.cc:194] Registered new tserver with Master: bd2ef38ff69b4e82bb5838016e975fa1 (127.29.62.193:32875)
I20260808 17:49:32.336730 27119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.193@KRBTEST.COM'} at 127.29.62.193:40147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
Entry for principal kudu/127.29.62.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab.
I20260808 17:49:32.353356 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:41707
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:32.435068 27389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:32.435302 27389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:32.435340 27389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:32.436460 27389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:32.436529 27389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:32.436569 27389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:49:32.437985 27389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.194.keytab
--principal=kudu/127.29.62.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:41707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:32.438257 27389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:32.438504 27389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:32.443567 27396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:32.443606 27398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:32.443722 27389 server_base.cc:1034] running on GCE node
W20260808 17:49:32.443578 27400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:32.443989 27389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:32.444249 27389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:32.445398 27389 hybrid_clock.cc:648] HybridClock initialized: now 1786211372445380 us; error 37 us; skew 500 ppm
Aug 08 17:49:32 dist-test-slave-tk9z krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211372, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:32.447834 27389 init.cc:377] Logged in from keytab as kudu/127.29.62.194@KRBTEST.COM (short username kudu)
I20260808 17:49:32.448304 27389 webserver.cc:460] Webserver started at http://127.29.62.194:45033/ using document root <none> and password file <none>
I20260808 17:49:32.448544 27389 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:32.448614 27389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:32.448776 27389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:32.449767 27389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "f893da87a2214538a57bcbecb207fa7e"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "e009d983affab9666a57db89264a9b27"
server_key_iv: "34a876ecd01eca8354a884b91ac459d4"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.450148 27389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "f893da87a2214538a57bcbecb207fa7e"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "e009d983affab9666a57db89264a9b27"
server_key_iv: "34a876ecd01eca8354a884b91ac459d4"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.451514 27389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 17:49:32.452216 27408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.452392 27389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:32.452494 27389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "f893da87a2214538a57bcbecb207fa7e"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "e009d983affab9666a57db89264a9b27"
server_key_iv: "34a876ecd01eca8354a884b91ac459d4"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.452579 27389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:32.461709 27389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:32.463615 27389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:32.463773 27389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:32.464003 27389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:32.464304 27389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:32.464359 27389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.464401 27389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:32.464429 27389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.470507 27389 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:34985
I20260808 17:49:32.470562 27554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:34985 every 8 connection(s)
I20260808 17:49:32.470938 27389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:49:32 dist-test-slave-tk9z krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211372, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.194@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:32.478753 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 27389
I20260808 17:49:32.478888 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:49:32.479089 29947 external_mini_cluster.cc:1428] Setting key ca23f3a985d0934c407df1a30c60b10d
I20260808 17:49:32.480731 27559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.471659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:41707 (local address 127.29.62.194:39735)
0808 17:49:32.471825 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.471867 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.472336 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.472542 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.472545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.473138 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:32.473330 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.473334 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.473984 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.473985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.474851 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.474854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.475186 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.475187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.475280 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.475827 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:32.475835 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:32.477623 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:32.479329 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:32.479335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:32.479337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:32.479559 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:32.479802 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:32.479804 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:32.479804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:32.479942 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:32.480234 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:32.480239 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:32.480372 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.480457 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.480573 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260808 17:49:32.480734 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.471683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:39735 (local address 127.29.62.254:41707)
0808 17:49:32.471752 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.471754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.471948 (+ 194us) server_negotiation.cc:407] Connection header received
0808 17:49:32.472423 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.472424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.472434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.472487 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:32.473409 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.473894 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.474935 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.475113 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.475165 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.477721 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:32.477722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:32.477723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:32.477729 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:32.479243 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.479641 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.479642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.479643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.479704 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.480019 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.480020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.480020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.480165 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:32.480238 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.480505 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.480593 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:49:32.481276 27556 heartbeater.cc:347] Connected to a master server at 127.29.62.254:41707
I20260808 17:49:32.481377 27556 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:32.481571 27556 heartbeater.cc:511] Master 127.29.62.254:41707 requested a full tablet report, sending...
I20260808 17:49:32.482016 27119 ts_manager.cc:194] Registered new tserver with Master: f893da87a2214538a57bcbecb207fa7e (127.29.62.194:34985)
I20260808 17:49:32.482574 27119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.194@KRBTEST.COM'} at 127.29.62.194:39735
WARNING: no policy specified for kudu/127.29.62.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
Entry for principal kudu/127.29.62.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab.
I20260808 17:49:32.503383 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:41707
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:32.584966 27564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:32.585220 27564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:32.585263 27564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:32.586448 27564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:32.586522 27564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:32.586567 27564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:49:32.588083 27564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.195.keytab
--principal=kudu/127.29.62.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:41707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:32.588364 27564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:32.588608 27564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:32.593808 27575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:32.593848 27571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:32.593835 27572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:32.594112 27564 server_base.cc:1034] running on GCE node
I20260808 17:49:32.594275 27564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:32.594542 27564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:32.595717 27564 hybrid_clock.cc:648] HybridClock initialized: now 1786211372595708 us; error 41 us; skew 500 ppm
Aug 08 17:49:32 dist-test-slave-tk9z krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211372, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:32.598220 27564 init.cc:377] Logged in from keytab as kudu/127.29.62.195@KRBTEST.COM (short username kudu)
I20260808 17:49:32.598766 27564 webserver.cc:460] Webserver started at http://127.29.62.195:41475/ using document root <none> and password file <none>
I20260808 17:49:32.598999 27564 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:32.599066 27564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:32.599200 27564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:32.600236 27564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "9fe1c934903b40c2aec23a092a54341e"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "713098235c1fcdf1790d95ea76e8259e"
server_key_iv: "1f6b4e762e10fd5073f2b179630af2d1"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.600611 27564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "9fe1c934903b40c2aec23a092a54341e"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "713098235c1fcdf1790d95ea76e8259e"
server_key_iv: "1f6b4e762e10fd5073f2b179630af2d1"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.601931 27564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:49:32.602715 27583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.602888 27564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:32.603000 27564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "9fe1c934903b40c2aec23a092a54341e"
format_stamp: "Formatted at 2026-08-08 17:49:32 on dist-test-slave-tk9z"
server_key: "713098235c1fcdf1790d95ea76e8259e"
server_key_iv: "1f6b4e762e10fd5073f2b179630af2d1"
server_key_version: "encryptionkey@0"
I20260808 17:49:32.603073 27564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:32.615444 27564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:32.617307 27564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:32.617480 27564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:32.617713 27564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:32.618034 27564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:32.618089 27564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.618132 27564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:32.618160 27564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:32.624173 27564 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:34349
I20260808 17:49:32.624219 27742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:34349 every 8 connection(s)
I20260808 17:49:32.624548 27564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:49:32.628597 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 27564
I20260808 17:49:32.628728 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:49:32.628911 29947 external_mini_cluster.cc:1428] Setting key 5b1ab2097635e7db5327bfc05cc20fb4
Aug 08 17:49:32 dist-test-slave-tk9z krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211372, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.195@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:32.633775 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.625270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:48537 (local address 127.29.62.254:41707)
0808 17:49:32.625340 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.625342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.625514 (+ 172us) server_negotiation.cc:407] Connection header received
0808 17:49:32.626029 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.626030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.626040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.626099 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:32.627180 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.627685 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.628636 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.628955 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.629052 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.631288 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:32.631289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:32.631290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:32.631296 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:32.632674 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.633018 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.633019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.633019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.633071 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.633282 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.633283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.633283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.633379 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:32.633428 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.633648 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.633711 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:49:32.633972 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.625207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:41707 (local address 127.29.62.195:48537)
0808 17:49:32.625373 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.625416 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.625943 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.626181 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.626184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.626894 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:32.627095 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.627100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.627765 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.627765 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.628541 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.628543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.629067 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.629068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.629230 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.629743 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:49:32.629752 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:32.631205 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:32.632742 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:32.632747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:32.632749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:32.632952 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:32.633127 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:49:32.633128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:49:32.633128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:49:32.633228 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:49:32.633433 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:32.633437 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:32.633542 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.633621 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.633732 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260808 17:49:32.634451 27743 heartbeater.cc:347] Connected to a master server at 127.29.62.254:41707
I20260808 17:49:32.634593 27743 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:32.634778 27743 heartbeater.cc:511] Master 127.29.62.254:41707 requested a full tablet report, sending...
I20260808 17:49:32.635139 27119 ts_manager.cc:194] Registered new tserver with Master: 9fe1c934903b40c2aec23a092a54341e (127.29.62.195:34349)
I20260808 17:49:32.635468 27119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.62.195@KRBTEST.COM'} at 127.29.62.195:48537
I20260808 17:49:32.640874 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:49:32.648043 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.642222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.29.62.254:41707)
0808 17:49:32.642299 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.642302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.642420 (+ 118us) server_negotiation.cc:407] Connection header received
0808 17:49:32.642501 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.642502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.642512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.642584 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:32.643147 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.643618 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.644348 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.644541 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.644615 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.645767 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:32.645768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:32.645769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:32.645776 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:32.646898 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.647242 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.647243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.647244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.647302 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:32.647508 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:32.647509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:32.647509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:32.647648 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:32.647758 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.647860 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.647945 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 17:49:32.649462 27119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:49:32.651188 27119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:49:32.657541 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.653288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:34985 (local address 127.0.0.1:42992)
0808 17:49:32.653402 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.653442 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.653499 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.653884 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.653887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.653889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:32.653987 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.653989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.655104 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.655105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.656475 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.656477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.657199 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.657200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.657281 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.657294 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.657389 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.657449 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":71,"threads_started":1}
I20260808 17:49:32.657663 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.653510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:34349 (local address 127.0.0.1:39288)
0808 17:49:32.653709 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.653720 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.653768 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.654109 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.654110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.654112 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:32.654146 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.654147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.655465 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.655466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.656574 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.656576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.657407 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.657408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.657497 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.657502 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.657538 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.657580 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":48,"threads_started":1}
I20260808 17:49:32.657663 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.653457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.29.62.193:32875)
0808 17:49:32.653651 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.653656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.653769 (+ 113us) server_negotiation.cc:407] Connection header received
0808 17:49:32.653885 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.653888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.653903 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.653983 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:32.654154 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.655368 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.656543 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.657274 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.657363 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.657424 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.657530 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.657578 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":85,"threads_started":1}
I20260808 17:49:32.657837 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.653512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42992 (local address 127.29.62.194:34985)
0808 17:49:32.653688 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.653694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.653703 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:49:32.653745 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.653748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.653765 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.654112 (+ 347us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:32.654189 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.655006 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.656574 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.657109 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.657199 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.657261 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.657683 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.657737 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260808 17:49:32.658005 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.653677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39288 (local address 127.29.62.195:34349)
0808 17:49:32.653897 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.653901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.653909 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:49:32.653970 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.653986 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.654002 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.654219 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:32.654290 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.655355 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.656655 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.657308 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.657769 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.657841 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.657863 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.657916 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":91,"threads_started":1}
I20260808 17:49:32.658627 27469 tablet_service.cc:1467] Processing CreateTablet for tablet 7981b2168889497fa1f226825c3e4eda (DEFAULT_TABLE table=test-table [id=cff980ca520a4fd8ab0997c3c14b6c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:32.658798 27649 tablet_service.cc:1467] Processing CreateTablet for tablet 7981b2168889497fa1f226825c3e4eda (DEFAULT_TABLE table=test-table [id=cff980ca520a4fd8ab0997c3c14b6c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:32.659055 27469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7981b2168889497fa1f226825c3e4eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:32.659158 27649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7981b2168889497fa1f226825c3e4eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:32.661818 27759 tablet_bootstrap.cc:492] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Bootstrap starting.
I20260808 17:49:32.661830 27760 tablet_bootstrap.cc:492] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: Bootstrap starting.
I20260808 17:49:32.662017 27752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.653331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:32875 (local address 127.0.0.1:34608)
0808 17:49:32.653692 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.653703 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.653796 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.653992 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.653994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.653996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:32.654058 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.654060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.655495 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.655496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.656447 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.656450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.657371 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.657372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.657446 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.657449 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.657502 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.657533 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":1,"thread_start_us":52,"threads_started":1}
I20260808 17:49:32.662598 27760 tablet_bootstrap.cc:654] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:32.662635 27759 tablet_bootstrap.cc:654] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:32.662798 27293 tablet_service.cc:1467] Processing CreateTablet for tablet 7981b2168889497fa1f226825c3e4eda (DEFAULT_TABLE table=test-table [id=cff980ca520a4fd8ab0997c3c14b6c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:32.663182 27293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7981b2168889497fa1f226825c3e4eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:32.663190 27760 log.cc:824] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:32.663192 27759 log.cc:824] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:32.663968 27760 tablet_bootstrap.cc:492] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: No bootstrap required, opened a new log
I20260808 17:49:32.664045 27760 ts_tablet_manager.cc:1397] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:49:32.666062 27760 raft_consensus.cc:348] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.666261 27760 raft_consensus.cc:374] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:32.666301 27760 raft_consensus.cc:729] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fe1c934903b40c2aec23a092a54341e, State: Initialized, Role: FOLLOWER
I20260808 17:49:32.666473 27760 consensus_queue.cc:260] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.666756 27760 ts_tablet_manager.cc:1428] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:49:32.666787 27759 tablet_bootstrap.cc:492] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: No bootstrap required, opened a new log
I20260808 17:49:32.666865 27759 ts_tablet_manager.cc:1397] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 17:49:32.666877 27743 heartbeater.cc:503] Master 127.29.62.254:41707 was elected leader, sending a full tablet report...
I20260808 17:49:32.668491 27759 raft_consensus.cc:348] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.668653 27759 raft_consensus.cc:374] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:32.668701 27759 raft_consensus.cc:729] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f893da87a2214538a57bcbecb207fa7e, State: Initialized, Role: FOLLOWER
I20260808 17:49:32.668814 27759 consensus_queue.cc:260] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.669167 27759 ts_tablet_manager.cc:1428] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:49:32.669198 27556 heartbeater.cc:503] Master 127.29.62.254:41707 was elected leader, sending a full tablet report...
I20260808 17:49:32.669377 27763 tablet_bootstrap.cc:492] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1: Bootstrap starting.
I20260808 17:49:32.670045 27763 tablet_bootstrap.cc:654] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:32.670622 27763 log.cc:824] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:32.671307 27763 tablet_bootstrap.cc:492] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1: No bootstrap required, opened a new log
I20260808 17:49:32.671360 27763 ts_tablet_manager.cc:1397] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:49:32.672672 27763 raft_consensus.cc:348] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.672801 27763 raft_consensus.cc:374] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:32.672849 27763 raft_consensus.cc:729] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd2ef38ff69b4e82bb5838016e975fa1, State: Initialized, Role: FOLLOWER
I20260808 17:49:32.672968 27763 consensus_queue.cc:260] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.673277 27763 ts_tablet_manager.cc:1428] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:49:32.673296 27381 heartbeater.cc:503] Master 127.29.62.254:41707 was elected leader, sending a full tablet report...
W20260808 17:49:32.721572 27558 tablet.cc:2369] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:49:32.793121 27764 raft_consensus.cc:482] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:32.793246 27764 raft_consensus.cc:504] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.793711 27764 leader_election.cc:288] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f893da87a2214538a57bcbecb207fa7e (127.29.62.194:34985), bd2ef38ff69b4e82bb5838016e975fa1 (127.29.62.193:32875)
I20260808 17:49:32.795480 27767 raft_consensus.cc:482] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:32.795608 27767 raft_consensus.cc:504] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.795986 27767 leader_election.cc:288] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f893da87a2214538a57bcbecb207fa7e (127.29.62.194:34985), 9fe1c934903b40c2aec23a092a54341e (127.29.62.195:34349)
I20260808 17:49:32.797061 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.793928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:32875 (local address 127.29.62.195:36663)
0808 17:49:32.793999 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.794013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.794065 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.794228 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.794229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.794230 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:32.794304 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.794306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.794997 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.794998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.795948 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.795951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.796768 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.796769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.796859 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.796864 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.796915 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.796961 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:49:32.797333 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.794043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:36663 (local address 127.29.62.193:32875)
0808 17:49:32.794098 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.794101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.794110 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:49:32.794141 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.794141 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.794150 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.794207 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:32.794409 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.794902 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.796070 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.796646 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.796715 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.796727 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.797236 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.797277 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:49:32.797343 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.794015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:38493 (local address 127.29.62.194:34985)
0808 17:49:32.794068 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.794071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.794258 (+ 187us) server_negotiation.cc:407] Connection header received
0808 17:49:32.794322 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.794323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.794334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.794480 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:32.794654 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.795115 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.796380 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.796977 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.797029 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.797037 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.797238 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.797276 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 17:49:32.797572 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.793928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:34985 (local address 127.29.62.195:38493)
0808 17:49:32.794150 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.794161 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.794261 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.794489 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.794490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.794492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:32.794577 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.794579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.795197 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.795198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.796281 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.796284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.797095 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.797096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.797167 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.797170 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.797443 (+ 273us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.797490 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":58,"thread_start_us":103,"threads_started":1}
I20260808 17:49:32.797607 27323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7981b2168889497fa1f226825c3e4eda" candidate_uuid: "9fe1c934903b40c2aec23a092a54341e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" is_pre_election: true
I20260808 17:49:32.797719 27323 raft_consensus.cc:2454] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fe1c934903b40c2aec23a092a54341e in term 0.
I20260808 17:49:32.797897 27501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7981b2168889497fa1f226825c3e4eda" candidate_uuid: "9fe1c934903b40c2aec23a092a54341e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f893da87a2214538a57bcbecb207fa7e" is_pre_election: true
I20260808 17:49:32.798041 27501 raft_consensus.cc:2454] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fe1c934903b40c2aec23a092a54341e in term 0.
I20260808 17:49:32.798252 27589 leader_election.cc:302] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fe1c934903b40c2aec23a092a54341e, bd2ef38ff69b4e82bb5838016e975fa1; no voters:
I20260808 17:49:32.798449 27764 raft_consensus.cc:2781] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:49:32.798512 27764 raft_consensus.cc:482] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:49:32.798537 27764 raft_consensus.cc:3037] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:32.799568 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.796221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:34349 (local address 127.29.62.193:51881)
0808 17:49:32.796410 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.796419 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.796464 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.796636 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.796637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.796639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:32.796705 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.796707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.797546 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.797547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.798715 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.798718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.799285 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.799286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.799356 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.799359 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.799444 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.799482 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260808 17:49:32.799561 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.796453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:51881 (local address 127.29.62.195:34349)
0808 17:49:32.796507 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.796509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.796516 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:49:32.796535 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.796536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.796544 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.796596 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:32.796775 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.797439 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.798809 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.799208 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.799283 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.799290 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.799457 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.799490 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:49:32.799479 27764 raft_consensus.cc:504] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.799865 27677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7981b2168889497fa1f226825c3e4eda" candidate_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fe1c934903b40c2aec23a092a54341e" is_pre_election: true
I20260808 17:49:32.800022 27677 raft_consensus.cc:2379] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd2ef38ff69b4e82bb5838016e975fa1 in current term 1: Already voted for candidate 9fe1c934903b40c2aec23a092a54341e in this term.
I20260808 17:49:32.800053 27323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7981b2168889497fa1f226825c3e4eda" candidate_uuid: "9fe1c934903b40c2aec23a092a54341e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd2ef38ff69b4e82bb5838016e975fa1"
I20260808 17:49:32.800133 27323 raft_consensus.cc:3037] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:32.800359 27501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7981b2168889497fa1f226825c3e4eda" candidate_uuid: "9fe1c934903b40c2aec23a092a54341e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f893da87a2214538a57bcbecb207fa7e"
I20260808 17:49:32.800431 27501 raft_consensus.cc:3037] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:32.800525 27385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.796087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:34985 (local address 127.29.62.193:42351)
0808 17:49:32.796127 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.796137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:32.796183 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:32.796837 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:32.796838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:32.796840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:32.796894 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.796895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.798430 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.798431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.799490 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:32.799493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.800271 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.800272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:32.800343 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.800346 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:32.800418 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 17:49:32.800463 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 17:49:32.800542 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.796295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:42351 (local address 127.29.62.194:34985)
0808 17:49:32.796609 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.796611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.796618 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:49:32.796641 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.796641 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.796651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.796956 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:32.797026 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.798310 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.799597 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.800197 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.800273 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.800281 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.800426 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.800463 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":289,"thread_start_us":117,"threads_started":1}
I20260808 17:49:32.801025 27323 raft_consensus.cc:2454] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fe1c934903b40c2aec23a092a54341e in term 1.
I20260808 17:49:32.801159 27498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7981b2168889497fa1f226825c3e4eda" candidate_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f893da87a2214538a57bcbecb207fa7e" is_pre_election: true
I20260808 17:49:32.801323 27501 raft_consensus.cc:2454] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fe1c934903b40c2aec23a092a54341e in term 1.
I20260808 17:49:32.801425 27589 leader_election.cc:302] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fe1c934903b40c2aec23a092a54341e, bd2ef38ff69b4e82bb5838016e975fa1; no voters:
I20260808 17:49:32.801491 27232 leader_election.cc:302] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd2ef38ff69b4e82bb5838016e975fa1; no voters: 9fe1c934903b40c2aec23a092a54341e, f893da87a2214538a57bcbecb207fa7e
I20260808 17:49:32.799873 27764 leader_election.cc:288] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [CANDIDATE]: Term 1 election: Requested vote from peers f893da87a2214538a57bcbecb207fa7e (127.29.62.194:34985), bd2ef38ff69b4e82bb5838016e975fa1 (127.29.62.193:32875)
I20260808 17:49:32.801663 27771 raft_consensus.cc:2781] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:32.801728 27771 raft_consensus.cc:686] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 1 LEADER]: Becoming Leader. State: Replica: 9fe1c934903b40c2aec23a092a54341e, State: Running, Role: LEADER
I20260808 17:49:32.801820 27771 consensus_queue.cc:237] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } }
I20260808 17:49:32.801914 27767 raft_consensus.cc:2726] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 17:49:32.802677 27119 catalog_manager.cc:5495] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fe1c934903b40c2aec23a092a54341e (127.29.62.195). New cstate: current_term: 1 leader_uuid: "9fe1c934903b40c2aec23a092a54341e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fe1c934903b40c2aec23a092a54341e" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 34349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 } health_report { overall_health: UNKNOWN } } }
W20260808 17:49:32.825271 27383 tablet.cc:2369] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:49:32.842484 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.839790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.29.62.195:34349)
0808 17:49:32.839846 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:32.839848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:32.839858 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:49:32.839941 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:32.839942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:32.839964 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:32.840054 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:49:32.840188 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.840750 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.841626 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:32.841826 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:32.841903 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:32.842057 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:49:32.842172 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:49:32.842294 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:32.842320 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 17:49:32.842339 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:49:32.844261 27501 raft_consensus.cc:1261] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 1 FOLLOWER]: Refusing update from remote peer 9fe1c934903b40c2aec23a092a54341e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:32.844359 27323 raft_consensus.cc:1261] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 1 FOLLOWER]: Refusing update from remote peer 9fe1c934903b40c2aec23a092a54341e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:32.844604 27764 consensus_queue.cc:1035] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f893da87a2214538a57bcbecb207fa7e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 34985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:32.844731 27771 consensus_queue.cc:1035] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd2ef38ff69b4e82bb5838016e975fa1" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:32.847297 27774 mvcc.cc:205] Tried to move back new op lower bound from 7316321783167512576 to 7316321782996946944. Current Snapshot: MvccSnapshot[applied={T|T < 7316321783167512576}]
I20260808 17:49:32.849515 27777 mvcc.cc:205] Tried to move back new op lower bound from 7316321783167512576 to 7316321782996946944. Current Snapshot: MvccSnapshot[applied={T|T < 7316321783167512576}]
I20260808 17:49:32.850708 27118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:32.850806 27118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:32.851603 27118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72df34ee7d994f89add0733ad78953a6: Sending DeleteTablet for 3 replicas of tablet 7981b2168889497fa1f226825c3e4eda
I20260808 17:49:32.851984 27293 tablet_service.cc:1514] Processing DeleteTablet for tablet 7981b2168889497fa1f226825c3e4eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:34608
I20260808 17:49:32.852012 27469 tablet_service.cc:1514] Processing DeleteTablet for tablet 7981b2168889497fa1f226825c3e4eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:42992
I20260808 17:49:32.852213 27649 tablet_service.cc:1514] Processing DeleteTablet for tablet 7981b2168889497fa1f226825c3e4eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:32 UTC) from {username='kudu', principal='kudu/127.29.62.254@KRBTEST.COM'} at 127.0.0.1:39288
I20260808 17:49:32.852260 27790 tablet_replica.cc:330] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: stopping tablet replica
I20260808 17:49:32.852437 27789 tablet_replica.cc:330] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1: stopping tablet replica
I20260808 17:49:32.852454 27790 raft_consensus.cc:2229] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:32.852540 27789 raft_consensus.cc:2229] T 7981b2168889497fa1f226825c3e4eda P bd2ef38ff69b4e82bb5838016e975fa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:32.852566 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 27205
I20260808 17:49:32.853205 27790 raft_consensus.cc:2258] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:32.855559 27790 ts_tablet_manager.cc:1905] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:32.858484 27790 ts_tablet_manager.cc:1918] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:32.858625 27790 log.cc:1196] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/7981b2168889497fa1f226825c3e4eda
I20260808 17:49:32.858952 27790 ts_tablet_manager.cc:1939] T 7981b2168889497fa1f226825c3e4eda P f893da87a2214538a57bcbecb207fa7e: Deleting consensus metadata
I20260808 17:49:32.859378 27101 catalog_manager.cc:4841] TS f893da87a2214538a57bcbecb207fa7e (127.29.62.194:34985): tablet 7981b2168889497fa1f226825c3e4eda (table test-table [id=cff980ca520a4fd8ab0997c3c14b6c57]) successfully deleted
I20260808 17:49:32.862838 27791 tablet_replica.cc:330] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: stopping tablet replica
I20260808 17:49:32.862959 27791 raft_consensus.cc:2229] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:49:32.863058 27791 raft_consensus.cc:2258] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:32.863988 27791 ts_tablet_manager.cc:1905] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:32.865473 27791 ts_tablet_manager.cc:1918] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:32.865533 27791 log.cc:1196] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/7981b2168889497fa1f226825c3e4eda
I20260808 17:49:32.865824 27791 ts_tablet_manager.cc:1939] T 7981b2168889497fa1f226825c3e4eda P 9fe1c934903b40c2aec23a092a54341e: Deleting consensus metadata
I20260808 17:49:32.866219 27101 catalog_manager.cc:4841] TS 9fe1c934903b40c2aec23a092a54341e (127.29.62.195:34349): tablet 7981b2168889497fa1f226825c3e4eda (table test-table [id=cff980ca520a4fd8ab0997c3c14b6c57]) successfully deleted
W20260808 17:49:32.870080 27100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:32875 (error 108)
I20260808 17:49:32.870618 27752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:32.870497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:32875 (local address 127.0.0.1:34612)
0808 17:49:32.870539 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:32.870577 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:32875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 17:49:32.870786 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 27389
W20260808 17:49:32.870824 27100 catalog_manager.cc:4584] TS bd2ef38ff69b4e82bb5838016e975fa1 (127.29.62.193:32875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7981b2168889497fa1f226825c3e4eda: Network error: Client connection negotiation failed: client connection to 127.29.62.193:32875: connect: Connection refused (error 111)
I20260808 17:49:32.883800 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 27564
I20260808 17:49:32.892503 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 27076
2026-08-08T17:49:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2328 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T17:49:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:32Z Disabled control of system clock
I20260808 17:49:32.928850 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:38091
--webserver_interface=127.29.62.254
--webserver_port=0
--builtin_ntp_servers=127.29.62.212:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:38091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:33.012952 27802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:33.013171 27802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:33.013192 27802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:33.014290 27802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:33.014325 27802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:33.014338 27802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:33.014351 27802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:33.014364 27802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:33.015751 27802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:38091
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:38091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:33.015964 27802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:33.016213 27802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:33.020890 27812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:33.020928 27810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:33.020928 27809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:33.021572 27802 server_base.cc:1034] running on GCE node
I20260808 17:49:33.021821 27802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:33.022109 27802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:33.023274 27802 hybrid_clock.cc:648] HybridClock initialized: now 1786211373023262 us; error 41 us; skew 500 ppm
I20260808 17:49:33.024612 27802 webserver.cc:460] Webserver started at http://127.29.62.254:43239/ using document root <none> and password file <none>
I20260808 17:49:33.024864 27802 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:33.024938 27802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:33.025103 27802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:33.026188 27802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "e5feb813af8746a8a75194bf883ea7e8"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "672afa5f3a383f7db2832233f80a6f29"
server_key_iv: "efafb52f4edb0c814d684a799c37b2d9"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.026599 27802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "e5feb813af8746a8a75194bf883ea7e8"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "672afa5f3a383f7db2832233f80a6f29"
server_key_iv: "efafb52f4edb0c814d684a799c37b2d9"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.027915 27802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:33.028623 27819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.028798 27802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:49:33.028910 27802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "e5feb813af8746a8a75194bf883ea7e8"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "672afa5f3a383f7db2832233f80a6f29"
server_key_iv: "efafb52f4edb0c814d684a799c37b2d9"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.028990 27802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:33.038013 27802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:33.038314 27802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:33.038491 27802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:33.042317 27802 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:38091
I20260808 17:49:33.042352 27891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:38091 every 8 connection(s)
I20260808 17:49:33.042760 27802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:33.043375 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 27802
I20260808 17:49:33.043514 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:33.043721 29947 external_mini_cluster.cc:1428] Setting key 4d00d0751012155798a90819d2204503
I20260808 17:49:33.043928 27892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:33.046857 27892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8: Bootstrap starting.
I20260808 17:49:33.047489 27892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:33.047787 27892 log.cc:824] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:33.048516 27892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8: No bootstrap required, opened a new log
I20260808 17:49:33.049278 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.044270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.29.62.254:38091)
0808 17:49:33.044501 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.044508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.044532 (+ 24us) server_negotiation.cc:407] Connection header received
0808 17:49:33.045018 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.045025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.045083 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.045232 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:33.045472 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.046354 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.047272 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.047753 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.047900 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.048616 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:33.048621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:33.048621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:33.048804 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:33.048838 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:33.048849 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:33.048925 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.049009 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.049088 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260808 17:49:33.050782 27892 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5feb813af8746a8a75194bf883ea7e8" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38091 } }
I20260808 17:49:33.050952 27892 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:33.051000 27892 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5feb813af8746a8a75194bf883ea7e8, State: Initialized, Role: FOLLOWER
I20260808 17:49:33.051146 27892 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5feb813af8746a8a75194bf883ea7e8" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38091 } }
I20260808 17:49:33.051254 27892 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:33.051302 27892 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:33.051350 27892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:33.051939 27892 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5feb813af8746a8a75194bf883ea7e8" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38091 } }
I20260808 17:49:33.052053 27892 leader_election.cc:302] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5feb813af8746a8a75194bf883ea7e8; no voters:
I20260808 17:49:33.052244 27892 leader_election.cc:288] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:33.052320 27900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:33.052467 27900 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [term 1 LEADER]: Becoming Leader. State: Replica: e5feb813af8746a8a75194bf883ea7e8, State: Running, Role: LEADER
I20260808 17:49:33.052536 27892 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:33.052556 27900 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5feb813af8746a8a75194bf883ea7e8" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38091 } }
I20260808 17:49:33.052951 27902 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5feb813af8746a8a75194bf883ea7e8. Latest consensus state: current_term: 1 leader_uuid: "e5feb813af8746a8a75194bf883ea7e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5feb813af8746a8a75194bf883ea7e8" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38091 } } }
I20260808 17:49:33.052953 27901 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5feb813af8746a8a75194bf883ea7e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5feb813af8746a8a75194bf883ea7e8" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 38091 } } }
I20260808 17:49:33.053071 27902 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:33.053074 27901 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8 [sys.catalog]: This master's current role is: LEADER
W20260808 17:49:33.054433 27921 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:49:33.054502 27921 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:49:33.054566 27913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:33.054771 27913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:33.056217 27913 catalog_manager.cc:1347] Generated new cluster ID: 9c1d44a13c9a48e490cc0bed0c91dc72
I20260808 17:49:33.056272 27913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:33.077376 27913 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:33.077847 27913 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:33.082021 27913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8: Generated new TSK 0
I20260808 17:49:33.082221 27913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:33.094355 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.193:0
--local_ip_for_outbound_sockets=127.29.62.193
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38091
--builtin_ntp_servers=127.29.62.212:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:33.178515 27924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:33.178725 27924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:33.178747 27924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:33.179939 27924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:33.179982 27924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:33.180009 27924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.193
I20260808 17:49:33.181408 27924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.62.193
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.193
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:33.181674 27924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:33.181988 27924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:33.187110 27941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:33.187107 27937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:33.187110 27936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:33.187340 27924 server_base.cc:1034] running on GCE node
I20260808 17:49:33.187639 27924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:33.187929 27924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:33.189090 27924 hybrid_clock.cc:648] HybridClock initialized: now 1786211373189078 us; error 26 us; skew 500 ppm
I20260808 17:49:33.190346 27924 webserver.cc:460] Webserver started at http://127.29.62.193:45097/ using document root <none> and password file <none>
I20260808 17:49:33.190619 27924 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:33.190665 27924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:33.190837 27924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:33.191936 27924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/data/instance:
uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "b41c73fe5efc2016b7acc7f6245b9204"
server_key_iv: "6b89eeb885f92546636158ff7f11abc3"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.192310 27924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance:
uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "b41c73fe5efc2016b7acc7f6245b9204"
server_key_iv: "6b89eeb885f92546636158ff7f11abc3"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.193764 27924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:33.194572 27950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.194765 27924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:33.194859 27924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/wal
uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "b41c73fe5efc2016b7acc7f6245b9204"
server_key_iv: "6b89eeb885f92546636158ff7f11abc3"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.194941 27924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:33.210738 27924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:33.211040 27924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:33.211182 27924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:33.211433 27924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:33.211819 27924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:33.211855 27924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.211879 27924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:33.211928 27924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.217905 27924 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.193:34649
I20260808 17:49:33.217983 28115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.193:34649 every 8 connection(s)
I20260808 17:49:33.218330 27924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/data/info.pb
I20260808 17:49:33.220458 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 27924
I20260808 17:49:33.220594 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-0/wal/instance
I20260808 17:49:33.220788 29947 external_mini_cluster.cc:1428] Setting key 9e3659d474d60a3c9d86eddc0e71b82e
I20260808 17:49:33.221791 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.194:0
--local_ip_for_outbound_sockets=127.29.62.194
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38091
--builtin_ntp_servers=127.29.62.212:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:49:33.224116 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.219298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:52335 (local address 127.29.62.254:38091)
0808 17:49:33.219413 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.219416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.219519 (+ 103us) server_negotiation.cc:407] Connection header received
0808 17:49:33.220033 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.220034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.220047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.220114 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:33.220870 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.221541 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.222592 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.222805 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.222878 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.223789 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:33.223790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:33.223791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:33.223797 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:33.223817 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:33.223822 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:33.223884 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.223977 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.224008 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 17:49:33.224228 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.219177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38091 (local address 127.29.62.193:52335)
0808 17:49:33.219376 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.219422 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.219948 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.220160 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.220163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.220171 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:33.220763 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.220770 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.221636 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.221637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.222500 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.222503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.222900 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.222901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.223034 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.223643 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:33.223643 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:33.223672 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:33.223696 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:33.223891 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:33.223895 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:33.223898 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.223958 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.224038 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260808 17:49:33.224717 28116 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38091
I20260808 17:49:33.224834 28116 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:33.225019 28116 heartbeater.cc:511] Master 127.29.62.254:38091 requested a full tablet report, sending...
I20260808 17:49:33.225571 27844 ts_manager.cc:194] Registered new tserver with Master: bbc32c9a8afc4362bb6bb6acb3dfc69d (127.29.62.193:34649)
I20260808 17:49:33.226544 27844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.193:52335
W20260808 17:49:33.305203 28121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:33.305423 28121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:33.305444 28121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:33.306581 28121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:33.306619 28121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:33.306644 28121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.194
I20260808 17:49:33.307928 28121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.62.194
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.194
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:33.308115 28121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:33.308360 28121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:33.313287 28128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:33.313277 28131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:33.313259 28127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:33.313531 28121 server_base.cc:1034] running on GCE node
I20260808 17:49:33.313738 28121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:33.313989 28121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:33.315140 28121 hybrid_clock.cc:648] HybridClock initialized: now 1786211373315139 us; error 39 us; skew 500 ppm
I20260808 17:49:33.316414 28121 webserver.cc:460] Webserver started at http://127.29.62.194:46691/ using document root <none> and password file <none>
I20260808 17:49:33.316664 28121 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:33.316732 28121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:33.316869 28121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:33.317932 28121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/data/instance:
uuid: "beaac515e1a14d7ba6d95f1f22f82f5e"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "aded44cd3643919758cbe9c2b9ec8826"
server_key_iv: "a1e18ee7e6d6b1e38027dc6173253ecd"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.318296 28121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance:
uuid: "beaac515e1a14d7ba6d95f1f22f82f5e"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "aded44cd3643919758cbe9c2b9ec8826"
server_key_iv: "a1e18ee7e6d6b1e38027dc6173253ecd"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.319618 28121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:33.320348 28139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.320551 28121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:49:33.320637 28121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/wal
uuid: "beaac515e1a14d7ba6d95f1f22f82f5e"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "aded44cd3643919758cbe9c2b9ec8826"
server_key_iv: "a1e18ee7e6d6b1e38027dc6173253ecd"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.320693 28121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:33.334051 28121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:33.334295 28121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:33.334465 28121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:33.334657 28121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:33.334933 28121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:33.334964 28121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.335004 28121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:33.335024 28121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.340580 28121 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.194:33257
I20260808 17:49:33.340664 28309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.194:33257 every 8 connection(s)
I20260808 17:49:33.340977 28121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/data/info.pb
I20260808 17:49:33.346166 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.341713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:47699 (local address 127.29.62.254:38091)
0808 17:49:33.341777 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.341779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.342016 (+ 237us) server_negotiation.cc:407] Connection header received
0808 17:49:33.342625 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.342626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.342640 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.342692 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:33.343373 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.343866 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.344834 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.345034 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.345113 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.345871 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:33.345873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:33.345873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:33.345878 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:33.345892 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:33.345896 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:33.345962 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.346058 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.346086 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:49:33.346405 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.341693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38091 (local address 127.29.62.194:47699)
0808 17:49:33.341896 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.341930 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.342516 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.342765 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.342769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.342777 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:33.343266 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.343270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.343963 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.343966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.344739 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.344741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.345121 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.345122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.345257 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.345730 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:33.345730 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:33.345759 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:33.345782 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:33.345962 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:33.345969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:33.345972 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.346060 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.346162 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260808 17:49:33.346910 28310 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38091
I20260808 17:49:33.347014 28310 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:33.347190 28310 heartbeater.cc:511] Master 127.29.62.254:38091 requested a full tablet report, sending...
I20260808 17:49:33.347592 27844 ts_manager.cc:194] Registered new tserver with Master: beaac515e1a14d7ba6d95f1f22f82f5e (127.29.62.194:33257)
I20260808 17:49:33.347908 27844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.194:47699
I20260808 17:49:33.348376 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 28121
I20260808 17:49:33.348497 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/wal/instance
I20260808 17:49:33.348726 29947 external_mini_cluster.cc:1428] Setting key 87c76ee71c69bbbd72e1c3e893c6a20c
I20260808 17:49:33.349583 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.62.195:0
--local_ip_for_outbound_sockets=127.29.62.195
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38091
--builtin_ntp_servers=127.29.62.212:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:49:33.432627 28313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:33.432827 28313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:33.432848 28313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:33.433964 28313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:33.434003 28313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:33.434053 28313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.62.195
I20260808 17:49:33.435478 28313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.62.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.62.195
--webserver_port=0
--tserver_master_addrs=127.29.62.254:38091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.62.195
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:33.435719 28313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:33.436009 28313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:33.441026 28322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:33.441047 28320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:33.441083 28313 server_base.cc:1034] running on GCE node
W20260808 17:49:33.441033 28324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:33.441512 28313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:33.441777 28313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:33.442934 28313 hybrid_clock.cc:648] HybridClock initialized: now 1786211373442922 us; error 48 us; skew 500 ppm
I20260808 17:49:33.444257 28313 webserver.cc:460] Webserver started at http://127.29.62.195:45971/ using document root <none> and password file <none>
I20260808 17:49:33.444504 28313 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:33.444576 28313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:33.444737 28313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:33.445824 28313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/data/instance:
uuid: "f9cbc468c7b1471595df7e6d1e8f4247"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "e702eabb93eee9fc148c16563ede6065"
server_key_iv: "af71e391466a22939f40dfa24c3cb664"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.446190 28313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance:
uuid: "f9cbc468c7b1471595df7e6d1e8f4247"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "e702eabb93eee9fc148c16563ede6065"
server_key_iv: "af71e391466a22939f40dfa24c3cb664"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.447533 28313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:33.448225 28332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.448392 28313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:49:33.448513 28313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/wal
uuid: "f9cbc468c7b1471595df7e6d1e8f4247"
format_stamp: "Formatted at 2026-08-08 17:49:33 on dist-test-slave-tk9z"
server_key: "e702eabb93eee9fc148c16563ede6065"
server_key_iv: "af71e391466a22939f40dfa24c3cb664"
server_key_version: "encryptionkey@0"
I20260808 17:49:33.448616 28313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:33.471438 28313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:33.471789 28313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:33.471956 28313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:33.472226 28313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:49:33.472589 28313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:49:33.472645 28313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.472689 28313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:49:33.472718 28313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:33.478550 28313 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.195:41935
I20260808 17:49:33.478592 28497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.195:41935 every 8 connection(s)
I20260808 17:49:33.478966 28313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/data/info.pb
I20260808 17:49:33.483966 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.479750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.195:56473 (local address 127.29.62.254:38091)
0808 17:49:33.479840 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.479843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.480033 (+ 190us) server_negotiation.cc:407] Connection header received
0808 17:49:33.480533 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.480534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.480545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.480600 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:33.481259 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.481747 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.482758 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.482941 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.482992 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.483741 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:33.483742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:33.483743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:33.483746 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:33.483760 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:33.483764 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:33.483806 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.483885 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.483906 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 17:49:33.484167 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.479704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:38091 (local address 127.29.62.195:56473)
0808 17:49:33.479906 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.479942 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.480446 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.480656 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.480659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.480668 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:49:33.481186 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.481191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.481825 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.481825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.482688 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.482690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.483022 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.483022 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.483128 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.483614 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:49:33.483614 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:49:33.483643 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:49:33.483666 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:49:33.483818 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:49:33.483822 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:49:33.483824 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.483876 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.483949 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
I20260808 17:49:33.484697 28498 heartbeater.cc:347] Connected to a master server at 127.29.62.254:38091
I20260808 17:49:33.484802 28498 heartbeater.cc:464] Registering TS with master...
I20260808 17:49:33.484987 28498 heartbeater.cc:511] Master 127.29.62.254:38091 requested a full tablet report, sending...
I20260808 17:49:33.485378 27844 ts_manager.cc:194] Registered new tserver with Master: f9cbc468c7b1471595df7e6d1e8f4247 (127.29.62.195:41935)
I20260808 17:49:33.485514 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 28313
I20260808 17:49:33.485638 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/wal/instance
I20260808 17:49:33.485834 29947 external_mini_cluster.cc:1428] Setting key cd28c091b9c4c3d63ea63c7c14f44a4f
I20260808 17:49:33.485821 27844 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.62.195:56473
I20260808 17:49:33.486740 29947 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:49:33.490569 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.488055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.29.62.254:38091)
0808 17:49:33.488119 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.488121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.488229 (+ 108us) server_negotiation.cc:407] Connection header received
0808 17:49:33.488264 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.488264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.488273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.488316 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:33.488428 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.488845 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.489525 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.489690 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.489737 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.490292 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:33.490293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:33.490294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:33.490297 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:33.490308 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:33.490312 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:33.490353 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.490450 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.490470 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:49:33.491912 27844 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:49:33.493425 27844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:49:33.499953 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.495672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48876 (local address 127.29.62.193:34649)
0808 17:49:33.495860 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.495865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.495876 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:49:33.495971 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.495973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.495989 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.496061 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:33.496301 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.497490 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.499074 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.499560 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.499649 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.499740 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.499831 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.499867 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260808 17:49:33.499943 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.495584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:34649 (local address 127.0.0.1:48876)
0808 17:49:33.495773 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.495817 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.495887 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.496148 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.496151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.496153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:33.496217 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.496219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.497600 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.497602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.498973 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.498975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.499649 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.499650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.499724 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.499736 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.499817 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.499867 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":29,"thread_start_us":64,"threads_started":1}
I20260808 17:49:33.500079 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.495806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33257 (local address 127.0.0.1:54066)
0808 17:49:33.496216 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.496227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.496276 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.496615 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.496616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.496618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:33.496663 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.496665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.497704 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.497704 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.498972 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.498975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.499864 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.499865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.499912 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.499915 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.499945 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.499985 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":83,"thread_start_us":68,"threads_started":1}
I20260808 17:49:33.500347 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.495939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.29.62.195:41935)
0808 17:49:33.496116 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.496120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.496131 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:49:33.496164 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.496166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.496180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.496256 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:33.496436 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.497511 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.498902 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.499625 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.499701 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.499799 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.500227 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.500269 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260808 17:49:33.500515 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.495607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:41935 (local address 127.0.0.1:34016)
0808 17:49:33.495880 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.495890 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.495941 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.496314 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.496315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.496317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:33.496352 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.496353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.497614 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.497616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.498814 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.498817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.499839 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.499841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.499903 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.499907 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.499958 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.499986 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":26,"thread_start_us":100,"threads_started":1}
I20260808 17:49:33.500525 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.496190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.29.62.194:33257)
0808 17:49:33.496442 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.496447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.496458 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:49:33.496497 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.496499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.496516 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.496743 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:33.496811 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.497622 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.499064 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.499773 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.500270 (+ 497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.500371 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.500399 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.500444 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":157,"threads_started":1}
I20260808 17:49:33.500705 28009 tablet_service.cc:1467] Processing CreateTablet for tablet 4dfc01f8379c48f1b748d578fd5f6aa6 (DEFAULT_TABLE table=test-table [id=0d873235e62d406e934a9e197548c2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:33.501109 28009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dfc01f8379c48f1b748d578fd5f6aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:33.501158 28218 tablet_service.cc:1467] Processing CreateTablet for tablet 4dfc01f8379c48f1b748d578fd5f6aa6 (DEFAULT_TABLE table=test-table [id=0d873235e62d406e934a9e197548c2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:33.501550 28218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dfc01f8379c48f1b748d578fd5f6aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:33.504019 28519 tablet_bootstrap.cc:492] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d: Bootstrap starting.
I20260808 17:49:33.504340 28520 tablet_bootstrap.cc:492] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Bootstrap starting.
I20260808 17:49:33.504649 28403 tablet_service.cc:1467] Processing CreateTablet for tablet 4dfc01f8379c48f1b748d578fd5f6aa6 (DEFAULT_TABLE table=test-table [id=0d873235e62d406e934a9e197548c2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:49:33.504858 28519 tablet_bootstrap.cc:654] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:33.504978 28520 tablet_bootstrap.cc:654] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:33.505017 28403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dfc01f8379c48f1b748d578fd5f6aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:33.505182 28519 log.cc:824] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:33.505336 28520 log.cc:824] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:33.506028 28519 tablet_bootstrap.cc:492] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d: No bootstrap required, opened a new log
I20260808 17:49:33.506104 28519 ts_tablet_manager.cc:1397] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:49:33.506842 28520 tablet_bootstrap.cc:492] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: No bootstrap required, opened a new log
I20260808 17:49:33.506927 28520 ts_tablet_manager.cc:1397] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 17:49:33.508435 28527 tablet_bootstrap.cc:492] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: Bootstrap starting.
I20260808 17:49:33.508752 28519 raft_consensus.cc:348] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.508884 28519 raft_consensus.cc:374] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:33.508939 28519 raft_consensus.cc:729] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc32c9a8afc4362bb6bb6acb3dfc69d, State: Initialized, Role: FOLLOWER
I20260808 17:49:33.509060 28519 consensus_queue.cc:260] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.509047 28520 raft_consensus.cc:348] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.509222 28520 raft_consensus.cc:374] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:33.509292 28520 raft_consensus.cc:729] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beaac515e1a14d7ba6d95f1f22f82f5e, State: Initialized, Role: FOLLOWER
I20260808 17:49:33.509330 28519 ts_tablet_manager.cc:1428] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 17:49:33.509320 28527 tablet_bootstrap.cc:654] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:33.509510 28116 heartbeater.cc:503] Master 127.29.62.254:38091 was elected leader, sending a full tablet report...
I20260808 17:49:33.509459 28520 consensus_queue.cc:260] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.509742 28520 ts_tablet_manager.cc:1428] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:49:33.509861 28310 heartbeater.cc:503] Master 127.29.62.254:38091 was elected leader, sending a full tablet report...
I20260808 17:49:33.510069 28527 log.cc:824] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:33.510836 28527 tablet_bootstrap.cc:492] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: No bootstrap required, opened a new log
I20260808 17:49:33.510890 28527 ts_tablet_manager.cc:1397] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 17:49:33.512612 28527 raft_consensus.cc:348] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.512732 28527 raft_consensus.cc:374] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:33.512799 28527 raft_consensus.cc:729] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9cbc468c7b1471595df7e6d1e8f4247, State: Initialized, Role: FOLLOWER
I20260808 17:49:33.512915 28527 consensus_queue.cc:260] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.513188 28527 ts_tablet_manager.cc:1428] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:49:33.513216 28498 heartbeater.cc:503] Master 127.29.62.254:38091 was elected leader, sending a full tablet report...
W20260808 17:49:33.591529 28311 tablet.cc:2369] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:49:33.646621 28528 raft_consensus.cc:482] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:33.646750 28528 raft_consensus.cc:504] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.647164 28529 raft_consensus.cc:482] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:49:33.647267 28528 leader_election.cc:288] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9cbc468c7b1471595df7e6d1e8f4247 (127.29.62.195:41935), beaac515e1a14d7ba6d95f1f22f82f5e (127.29.62.194:33257)
I20260808 17:49:33.647285 28529 raft_consensus.cc:504] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.647652 28529 leader_election.cc:288] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbc32c9a8afc4362bb6bb6acb3dfc69d (127.29.62.193:34649), f9cbc468c7b1471595df7e6d1e8f4247 (127.29.62.195:41935)
I20260808 17:49:33.651221 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.647598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:50281 (local address 127.29.62.194:33257)
0808 17:49:33.647639 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.647640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.647740 (+ 100us) server_negotiation.cc:407] Connection header received
0808 17:49:33.647850 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.647851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.647860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.647888 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:33.648107 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.648805 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.650288 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.650831 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.650881 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.650890 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.651112 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.651149 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:49:33.651221 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.647521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.193:44955 (local address 127.29.62.195:41935)
0808 17:49:33.647582 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.647584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.647747 (+ 163us) server_negotiation.cc:407] Connection header received
0808 17:49:33.647782 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.647783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.647793 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.647851 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:33.648035 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.648811 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.650277 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.650831 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.650897 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.650903 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.651112 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.651149 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:49:33.651475 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.647449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:41935 (local address 127.29.62.193:44955)
0808 17:49:33.647619 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.647632 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.647688 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.647855 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.647858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.647859 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:33.647939 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.647942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.648901 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.648903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.650164 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.650167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.650937 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.650938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.651011 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.651014 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.651064 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.651395 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":286}
I20260808 17:49:33.651484 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.647495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.194:33257 (local address 127.29.62.193:50281)
0808 17:49:33.647671 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.647683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.647771 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.647963 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.647963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.647965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:33.648030 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.648031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.648910 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.648910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.650164 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.650167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.650970 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.650971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.651030 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.651034 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.651351 (+ 317us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.651396 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":77,"thread_start_us":104,"threads_started":1}
I20260808 17:49:33.651831 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dfc01f8379c48f1b748d578fd5f6aa6" candidate_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" is_pre_election: true
I20260808 17:49:33.651880 28247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dfc01f8379c48f1b748d578fd5f6aa6" candidate_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" is_pre_election: true
I20260808 17:49:33.651995 28247 raft_consensus.cc:2454] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbc32c9a8afc4362bb6bb6acb3dfc69d in term 0.
I20260808 17:49:33.651997 28430 raft_consensus.cc:2454] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbc32c9a8afc4362bb6bb6acb3dfc69d in term 0.
I20260808 17:49:33.652052 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.648146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:57571 (local address 127.29.62.193:34649)
0808 17:49:33.648676 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.648678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.648684 (+ 6us) server_negotiation.cc:407] Connection header received
0808 17:49:33.648703 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.648704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.648713 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.648911 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:33.648942 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.649731 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.651004 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.651605 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.651926 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.651934 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.651955 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.651988 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":507}
I20260808 17:49:33.652210 27951 leader_election.cc:302] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbc32c9a8afc4362bb6bb6acb3dfc69d, beaac515e1a14d7ba6d95f1f22f82f5e; no voters:
I20260808 17:49:33.652365 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.647767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:34649 (local address 127.29.62.194:57571)
0808 17:49:33.647811 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.647820 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.647854 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.648789 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.648790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.648792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:33.648851 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.648853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.649829 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.649830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.650913 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.650915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.651713 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.651715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.651783 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.651787 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.651817 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.651857 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:49:33.652354 28528 raft_consensus.cc:2781] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:49:33.652411 28528 raft_consensus.cc:482] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:49:33.652438 28528 raft_consensus.cc:3037] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:33.652748 28041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dfc01f8379c48f1b748d578fd5f6aa6" candidate_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" is_pre_election: true
I20260808 17:49:33.652798 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.648047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.62.194:43971 (local address 127.29.62.195:41935)
0808 17:49:33.648190 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.648192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.648289 (+ 97us) server_negotiation.cc:407] Connection header received
0808 17:49:33.648334 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.648335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.648344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.648424 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:49:33.648604 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.649410 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.650914 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.652388 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.652499 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.652508 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.652689 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.652722 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260808 17:49:33.652803 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.647987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.195:41935 (local address 127.29.62.194:43971)
0808 17:49:33.648211 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.648220 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:49:33.648279 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:49:33.648442 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:49:33.648444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:49:33.648445 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:49:33.648533 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.648535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.649504 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.649506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.650815 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:49:33.650818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.652553 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.652554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:49:33.652623 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.652628 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:49:33.652684 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 17:49:33.652721 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
I20260808 17:49:33.653285 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dfc01f8379c48f1b748d578fd5f6aa6" candidate_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" is_pre_election: true
I20260808 17:49:33.653375 28430 raft_consensus.cc:2454] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beaac515e1a14d7ba6d95f1f22f82f5e in term 0.
I20260808 17:49:33.653443 28528 raft_consensus.cc:504] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.653590 28528 leader_election.cc:288] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [CANDIDATE]: Term 1 election: Requested vote from peers f9cbc468c7b1471595df7e6d1e8f4247 (127.29.62.195:41935), beaac515e1a14d7ba6d95f1f22f82f5e (127.29.62.194:33257)
I20260808 17:49:33.653566 28143 leader_election.cc:302] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beaac515e1a14d7ba6d95f1f22f82f5e, f9cbc468c7b1471595df7e6d1e8f4247; no voters:
I20260808 17:49:33.653776 28041 raft_consensus.cc:2379] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate beaac515e1a14d7ba6d95f1f22f82f5e in current term 1: Already voted for candidate bbc32c9a8afc4362bb6bb6acb3dfc69d in this term.
I20260808 17:49:33.653749 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dfc01f8379c48f1b748d578fd5f6aa6" candidate_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9cbc468c7b1471595df7e6d1e8f4247"
I20260808 17:49:33.653752 28529 raft_consensus.cc:2781] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:49:33.653776 28247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dfc01f8379c48f1b748d578fd5f6aa6" candidate_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e"
I20260808 17:49:33.653817 28529 raft_consensus.cc:482] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:49:33.653818 28430 raft_consensus.cc:3037] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:33.653858 28529 raft_consensus.cc:3037] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:33.654716 28430 raft_consensus.cc:2454] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbc32c9a8afc4362bb6bb6acb3dfc69d in term 1.
I20260808 17:49:33.654790 28529 raft_consensus.cc:504] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.654892 27953 leader_election.cc:302] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbc32c9a8afc4362bb6bb6acb3dfc69d, f9cbc468c7b1471595df7e6d1e8f4247; no voters:
I20260808 17:49:33.654969 28247 raft_consensus.cc:2379] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bbc32c9a8afc4362bb6bb6acb3dfc69d in current term 1: Already voted for candidate beaac515e1a14d7ba6d95f1f22f82f5e in this term.
I20260808 17:49:33.655004 28528 raft_consensus.cc:2781] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:33.654973 28529 leader_election.cc:288] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [CANDIDATE]: Term 1 election: Requested vote from peers bbc32c9a8afc4362bb6bb6acb3dfc69d (127.29.62.193:34649), f9cbc468c7b1471595df7e6d1e8f4247 (127.29.62.195:41935)
I20260808 17:49:33.655153 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dfc01f8379c48f1b748d578fd5f6aa6" candidate_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9cbc468c7b1471595df7e6d1e8f4247"
I20260808 17:49:33.655179 28041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dfc01f8379c48f1b748d578fd5f6aa6" candidate_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d"
I20260808 17:49:33.655252 28430 raft_consensus.cc:2379] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate beaac515e1a14d7ba6d95f1f22f82f5e in current term 1: Already voted for candidate bbc32c9a8afc4362bb6bb6acb3dfc69d in this term.
I20260808 17:49:33.655176 28528 raft_consensus.cc:686] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 1 LEADER]: Becoming Leader. State: Replica: bbc32c9a8afc4362bb6bb6acb3dfc69d, State: Running, Role: LEADER
I20260808 17:49:33.655436 28528 consensus_queue.cc:237] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } }
I20260808 17:49:33.656008 28140 leader_election.cc:302] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: beaac515e1a14d7ba6d95f1f22f82f5e; no voters: bbc32c9a8afc4362bb6bb6acb3dfc69d, f9cbc468c7b1471595df7e6d1e8f4247
I20260808 17:49:33.656123 28529 raft_consensus.cc:2726] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 17:49:33.656423 27844 catalog_manager.cc:5495] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d reported cstate change: term changed from 0 to 1, leader changed from <none> to bbc32c9a8afc4362bb6bb6acb3dfc69d (127.29.62.193). New cstate: current_term: 1 leader_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbc32c9a8afc4362bb6bb6acb3dfc69d" member_type: VOTER last_known_addr { host: "127.29.62.193" port: 34649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 } health_report { overall_health: UNKNOWN } } }
I20260808 17:49:33.683210 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.680651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48880 (local address 127.29.62.193:34649)
0808 17:49:33.680723 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:33.680725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:33.680736 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:49:33.680834 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:33.680835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:33.680856 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:33.680936 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:49:33.681095 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.681574 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.682328 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:33.682556 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:33.682638 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:33.682822 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:49:33.682931 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:49:33.683039 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:33.683064 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 17:49:33.683086 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:49:33.685132 28247 raft_consensus.cc:1261] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 1 FOLLOWER]: Refusing update from remote peer bbc32c9a8afc4362bb6bb6acb3dfc69d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:33.685139 28430 raft_consensus.cc:1261] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 1 FOLLOWER]: Refusing update from remote peer bbc32c9a8afc4362bb6bb6acb3dfc69d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:49:33.685429 28543 consensus_queue.cc:1035] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [LEADER]: Connected to new peer: Peer: permanent_uuid: "beaac515e1a14d7ba6d95f1f22f82f5e" member_type: VOTER last_known_addr { host: "127.29.62.194" port: 33257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:33.685578 28528 consensus_queue.cc:1035] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9cbc468c7b1471595df7e6d1e8f4247" member_type: VOTER last_known_addr { host: "127.29.62.195" port: 41935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:49:33.688264 28547 mvcc.cc:205] Tried to move back new op lower bound from 7316321786611486720 to 7316321786493280256. Current Snapshot: MvccSnapshot[applied={T|T < 7316321786611486720}]
I20260808 17:49:33.688707 28548 mvcc.cc:205] Tried to move back new op lower bound from 7316321786611486720 to 7316321786493280256. Current Snapshot: MvccSnapshot[applied={T|T < 7316321786611486720}]
I20260808 17:49:33.690984 27843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:33.691097 27843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:49:33.691897 27843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5feb813af8746a8a75194bf883ea7e8: Sending DeleteTablet for 3 replicas of tablet 4dfc01f8379c48f1b748d578fd5f6aa6
I20260808 17:49:33.692204 28009 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dfc01f8379c48f1b748d578fd5f6aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:33 UTC) from {username='slave'} at 127.0.0.1:48876
I20260808 17:49:33.692242 28403 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dfc01f8379c48f1b748d578fd5f6aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:33 UTC) from {username='slave'} at 127.0.0.1:34016
I20260808 17:49:33.692272 28218 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dfc01f8379c48f1b748d578fd5f6aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:49:33 UTC) from {username='slave'} at 127.0.0.1:54066
I20260808 17:49:33.692502 28562 tablet_replica.cc:330] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: stopping tablet replica
I20260808 17:49:33.692569 28560 tablet_replica.cc:330] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d: stopping tablet replica
I20260808 17:49:33.692674 28560 raft_consensus.cc:2229] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:49:33.692797 28560 raft_consensus.cc:2258] T 4dfc01f8379c48f1b748d578fd5f6aa6 P bbc32c9a8afc4362bb6bb6acb3dfc69d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:33.692907 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 27924
I20260808 17:49:33.701836 28562 raft_consensus.cc:2229] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:33.702018 28562 raft_consensus.cc:2258] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:33.702036 28563 tablet_replica.cc:330] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: stopping tablet replica
I20260808 17:49:33.702154 28563 raft_consensus.cc:2229] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:49:33.702266 28563 raft_consensus.cc:2258] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:49:33.702590 28562 ts_tablet_manager.cc:1905] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:33.702750 28563 ts_tablet_manager.cc:1905] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:49:33.704140 28562 ts_tablet_manager.cc:1918] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:33.704236 28562 log.cc:1196] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-1/wal/wals/4dfc01f8379c48f1b748d578fd5f6aa6
I20260808 17:49:33.704744 28562 ts_tablet_manager.cc:1939] T 4dfc01f8379c48f1b748d578fd5f6aa6 P beaac515e1a14d7ba6d95f1f22f82f5e: Deleting consensus metadata
I20260808 17:49:33.704731 28563 ts_tablet_manager.cc:1918] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:49:33.704833 28563 log.cc:1196] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: Deleting WAL directory at /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786211125631729-29947-0/minicluster-data/ts-2/wal/wals/4dfc01f8379c48f1b748d578fd5f6aa6
I20260808 17:49:33.705099 28563 ts_tablet_manager.cc:1939] T 4dfc01f8379c48f1b748d578fd5f6aa6 P f9cbc468c7b1471595df7e6d1e8f4247: Deleting consensus metadata
I20260808 17:49:33.705228 27821 catalog_manager.cc:4841] TS beaac515e1a14d7ba6d95f1f22f82f5e (127.29.62.194:33257): tablet 4dfc01f8379c48f1b748d578fd5f6aa6 (table test-table [id=0d873235e62d406e934a9e197548c2a3]) successfully deleted
I20260808 17:49:33.705506 27824 catalog_manager.cc:4841] TS f9cbc468c7b1471595df7e6d1e8f4247 (127.29.62.195:41935): tablet 4dfc01f8379c48f1b748d578fd5f6aa6 (table test-table [id=0d873235e62d406e934a9e197548c2a3]) successfully deleted
W20260808 17:49:33.707077 27821 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.62.193:34649 (error 108)
I20260808 17:49:33.707340 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 28121
I20260808 17:49:33.708961 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:33.707432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.193:34649 (local address 127.0.0.1:48890)
0808 17:49:33.708876 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0808 17:49:33.708932 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.193:34649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1417}
W20260808 17:49:33.709123 27821 catalog_manager.cc:4584] TS bbc32c9a8afc4362bb6bb6acb3dfc69d (127.29.62.193:34649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4dfc01f8379c48f1b748d578fd5f6aa6: Network error: Client connection negotiation failed: client connection to 127.29.62.193:34649: connect: Connection refused (error 111)
I20260808 17:49:33.716344 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 28313
I20260808 17:49:33.730158 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 27802
2026-08-08T17:49:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (833 ms)
[----------] 6 tests from EncryptionPolicyTest (9002 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:33 dist-test-slave-tk9z krb5kdc[28567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:33 dist-test-slave-tk9z krb5kdc[28567](info): set up 2 sockets
Aug 08 17:49:33 dist-test-slave-tk9z krb5kdc[28567](info): commencing operation
krb5kdc: starting...
W20260808 17:49:34.969238 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:35 dist-test-slave-tk9z krb5kdc[28567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:35.089015 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:35.171346 28585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:35.171558 28585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:35.171581 28585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:35.172745 28585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:35.172796 28585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:35.172811 28585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:35.172823 28585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:35.172878 28585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:35.174513 28585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39429
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:35.174778 28585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:35.175021 28585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:35.179930 28594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:35.179960 28591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:35.180032 28585 server_base.cc:1034] running on GCE node
W20260808 17:49:35.179955 28590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:35.180374 28585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:35.180673 28585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:35.181813 28585 hybrid_clock.cc:648] HybridClock initialized: now 1786211375181800 us; error 27 us; skew 500 ppm
Aug 08 17:49:35 dist-test-slave-tk9z krb5kdc[28567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:35.184311 28585 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:35.184746 28585 webserver.cc:460] Webserver started at http://127.0.0.1:34257/ using document root <none> and password file <none>
I20260808 17:49:35.184973 28585 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:35.185041 28585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:35.185214 28585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:35.186419 28585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "c20d0ccc02b44e5b803a955675fea498"
format_stamp: "Formatted at 2026-08-08 17:49:35 on dist-test-slave-tk9z"
server_key: "22668f004f9e50148ddcc192c97a49b8"
server_key_iv: "e8c6cbc6b602c8ebeb16829a28225e35"
server_key_version: "encryptionkey@0"
I20260808 17:49:35.186821 28585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "c20d0ccc02b44e5b803a955675fea498"
format_stamp: "Formatted at 2026-08-08 17:49:35 on dist-test-slave-tk9z"
server_key: "22668f004f9e50148ddcc192c97a49b8"
server_key_iv: "e8c6cbc6b602c8ebeb16829a28225e35"
server_key_version: "encryptionkey@0"
I20260808 17:49:35.188207 28585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:35.189029 28602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:35.189232 28585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:35.189338 28585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "c20d0ccc02b44e5b803a955675fea498"
format_stamp: "Formatted at 2026-08-08 17:49:35 on dist-test-slave-tk9z"
server_key: "22668f004f9e50148ddcc192c97a49b8"
server_key_iv: "e8c6cbc6b602c8ebeb16829a28225e35"
server_key_version: "encryptionkey@0"
I20260808 17:49:35.189416 28585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:35.213631 28585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:35.215935 28585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:35.216112 28585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:35.220185 28585 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39429
I20260808 17:49:35.220242 28674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39429 every 8 connection(s)
I20260808 17:49:35.220605 28585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:35.221557 28676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:35.223935 28676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498: Bootstrap starting.
I20260808 17:49:35.224376 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 28585
I20260808 17:49:35.224505 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:35.224552 28676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:35.224712 29947 external_mini_cluster.cc:1428] Setting key 084ca52a65b47a3ea7f6ebb8e3506392
I20260808 17:49:35.224891 28676 log.cc:824] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:35.225741 28676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498: No bootstrap required, opened a new log
I20260808 17:49:35.228299 28676 raft_consensus.cc:348] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20d0ccc02b44e5b803a955675fea498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260808 17:49:35.228482 28676 raft_consensus.cc:374] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:35.228518 28676 raft_consensus.cc:729] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20d0ccc02b44e5b803a955675fea498, State: Initialized, Role: FOLLOWER
I20260808 17:49:35.228684 28676 consensus_queue.cc:260] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20d0ccc02b44e5b803a955675fea498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260808 17:49:35.228762 28676 raft_consensus.cc:388] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:35.228785 28676 raft_consensus.cc:482] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:35.228859 28676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:35.229466 28676 raft_consensus.cc:504] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20d0ccc02b44e5b803a955675fea498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260808 17:49:35.229574 28676 leader_election.cc:302] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c20d0ccc02b44e5b803a955675fea498; no voters:
I20260808 17:49:35.229737 28676 leader_election.cc:288] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:35.229799 28684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:35.230010 28676 sys_catalog.cc:564] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:35.230032 28684 raft_consensus.cc:686] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [term 1 LEADER]: Becoming Leader. State: Replica: c20d0ccc02b44e5b803a955675fea498, State: Running, Role: LEADER
I20260808 17:49:35.230269 28684 consensus_queue.cc:237] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20d0ccc02b44e5b803a955675fea498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260808 17:49:35.230669 28686 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c20d0ccc02b44e5b803a955675fea498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20d0ccc02b44e5b803a955675fea498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } } }
I20260808 17:49:35.230758 28686 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:35.230986 28687 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c20d0ccc02b44e5b803a955675fea498. Latest consensus state: current_term: 1 leader_uuid: "c20d0ccc02b44e5b803a955675fea498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20d0ccc02b44e5b803a955675fea498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } } }
I20260808 17:49:35.231056 28687 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:35.231190 28698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:35.231709 28698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:49:35 dist-test-slave-tk9z krb5kdc[28567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:49:35.234251 28698 catalog_manager.cc:1347] Generated new cluster ID: c1b11d866896417eba747f179b1d3e41
I20260808 17:49:35.234308 28698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:35.241238 28698 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:35.241498 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:35.225288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36835 (local address 127.0.0.1:39429)
0808 17:49:35.225459 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:35.225469 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:35.225518 (+ 49us) server_negotiation.cc:407] Connection header received
0808 17:49:35.226162 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:35.226171 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:35.226234 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:35.226416 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:35.227079 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:35.228515 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:35.229409 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:35.230061 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:35.230575 (+ 514us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:35.233206 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:35.233211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:35.233211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:35.233224 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:35.239386 (+ 6162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:35.239925 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:35.239929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:35.239929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:35.240044 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:35.240305 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:35.240306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:35.240306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:35.240608 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:35.240776 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:35.241072 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 17:49:35.241235 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260808 17:49:35.241880 28698 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:35.242667 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:35.244851 28698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c20d0ccc02b44e5b803a955675fea498: Generated new TSK 0
I20260808 17:49:35.245081 28698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:35.250804 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:35.244216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37755 (local address 127.0.0.1:39429)
0808 17:49:35.244252 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:35.244254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:35.244440 (+ 186us) server_negotiation.cc:407] Connection header received
0808 17:49:35.244486 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:35.244486 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:35.244497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:35.244668 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:35.245341 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:35.245849 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:35.246692 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:35.246902 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:35.246974 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:35.248208 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:35.248210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:35.248210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:35.248217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:35.249580 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:35.249964 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:35.249966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:35.249966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:35.250019 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:35.250261 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:35.250262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:35.250262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:35.250417 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:35.250516 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:35.250646 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 17:49:35.250713 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:49:35.252578 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 28585
W20260808 17:49:35.269304 28565 connection.cc:535] client connection to 127.0.0.1:39429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39429): Connection reset by peer (error 104)
2026-08-08T17:49:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:35 dist-test-slave-tk9z krb5kdc[28722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:35 dist-test-slave-tk9z krb5kdc[28722](info): set up 2 sockets
Aug 08 17:49:35 dist-test-slave-tk9z krb5kdc[28722](info): commencing operation
krb5kdc: starting...
W20260808 17:49:36.506623 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:36 dist-test-slave-tk9z krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:36.628705 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:36.713760 28739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:36.714010 28739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:36.714049 28739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:36.715219 28739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:36.715289 28739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:36.715319 28739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:36.715348 28739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:36.715377 28739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:36.716810 28739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:36.717078 28739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:36.717301 28739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:36.722175 28747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:36.722244 28739 server_base.cc:1034] running on GCE node
W20260808 17:49:36.722443 28752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:36.722517 28748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:36.722807 28739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:36.723115 28739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:36.724310 28739 hybrid_clock.cc:648] HybridClock initialized: now 1786211376724294 us; error 29 us; skew 500 ppm
Aug 08 17:49:36 dist-test-slave-tk9z krb5kdc[28722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:36.731420 28739 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:36.731921 28739 webserver.cc:460] Webserver started at http://127.0.0.1:33743/ using document root <none> and password file <none>
I20260808 17:49:36.732115 28739 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:36.732156 28739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:36.732251 28739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:36.733290 28739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "3ed3d0f3a724499db3d2b1e837ee16fd"
format_stamp: "Formatted at 2026-08-08 17:49:36 on dist-test-slave-tk9z"
server_key: "876c88f2fc7c7978fdbed41fde9c5397"
server_key_iv: "f76684179b182d0533c81346eb6b8ee0"
server_key_version: "encryptionkey@0"
I20260808 17:49:36.733620 28739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "3ed3d0f3a724499db3d2b1e837ee16fd"
format_stamp: "Formatted at 2026-08-08 17:49:36 on dist-test-slave-tk9z"
server_key: "876c88f2fc7c7978fdbed41fde9c5397"
server_key_iv: "f76684179b182d0533c81346eb6b8ee0"
server_key_version: "encryptionkey@0"
I20260808 17:49:36.735050 28739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:36.735796 28768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:36.736008 28739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:36.736104 28739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "3ed3d0f3a724499db3d2b1e837ee16fd"
format_stamp: "Formatted at 2026-08-08 17:49:36 on dist-test-slave-tk9z"
server_key: "876c88f2fc7c7978fdbed41fde9c5397"
server_key_iv: "f76684179b182d0533c81346eb6b8ee0"
server_key_version: "encryptionkey@0"
I20260808 17:49:36.736162 28739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:36.766095 28739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:36.768661 28739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:36.768785 28739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:36.772779 28739 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36279
I20260808 17:49:36.772886 28835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36279 every 8 connection(s)
I20260808 17:49:36.773219 28739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:36.774303 28836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:36.774325 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 28739
I20260808 17:49:36.774497 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:36.774711 29947 external_mini_cluster.cc:1428] Setting key ad46a2d8d6565352d794fe35f4b679bd
I20260808 17:49:36.777894 28836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd: Bootstrap starting.
I20260808 17:49:36.778659 28836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:36.778959 28836 log.cc:824] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:36.779732 28836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd: No bootstrap required, opened a new log
I20260808 17:49:36.780591 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:36.775365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51629 (local address 127.0.0.1:36279)
0808 17:49:36.775570 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:36.775578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:36.775605 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:49:36.776062 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:36.776068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:36.776116 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:36.776231 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:36.776471 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:36.777364 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:36.778511 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:36.778962 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:36.779073 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:36.779806 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:36.779811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:36.779811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:36.780031 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:36.780072 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:36.780086 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:36.780182 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:36.780329 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 17:49:36.780360 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260808 17:49:36.781921 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:36.782227 28836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed3d0f3a724499db3d2b1e837ee16fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } }
I20260808 17:49:36.782450 28836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:36.782491 28836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed3d0f3a724499db3d2b1e837ee16fd, State: Initialized, Role: FOLLOWER
I20260808 17:49:36.782620 28836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed3d0f3a724499db3d2b1e837ee16fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } }
I20260808 17:49:36.782711 28836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:36.782747 28836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:36.782788 28836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:36.783572 28836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed3d0f3a724499db3d2b1e837ee16fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } }
I20260808 17:49:36.783716 28836 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed3d0f3a724499db3d2b1e837ee16fd; no voters:
I20260808 17:49:36.783890 28836 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:36.783927 28852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:36.784101 28852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [term 1 LEADER]: Becoming Leader. State: Replica: 3ed3d0f3a724499db3d2b1e837ee16fd, State: Running, Role: LEADER
I20260808 17:49:36.784180 28836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:36.784307 28852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed3d0f3a724499db3d2b1e837ee16fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } }
I20260808 17:49:36.784664 28852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed3d0f3a724499db3d2b1e837ee16fd. Latest consensus state: current_term: 1 leader_uuid: "3ed3d0f3a724499db3d2b1e837ee16fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed3d0f3a724499db3d2b1e837ee16fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } } }
I20260808 17:49:36.784678 28857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ed3d0f3a724499db3d2b1e837ee16fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed3d0f3a724499db3d2b1e837ee16fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } } }
I20260808 17:49:36.784811 28852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:36.784832 28857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:36.785032 28867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:36.785544 28867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:36.787390 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:36.783573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54513 (local address 127.0.0.1:36279)
0808 17:49:36.783632 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:36.783634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:36.783702 (+ 68us) server_negotiation.cc:407] Connection header received
0808 17:49:36.783779 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:36.783780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:36.783791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:36.783838 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:36.784063 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:36.784795 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:36.785924 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:36.786225 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:36.786294 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:36.787165 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:36.787166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:36.787166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:36.787171 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:36.787184 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:36.787188 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:36.787236 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:36.787307 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 17:49:36.787315 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:49:36.794157 28867 catalog_manager.cc:1347] Generated new cluster ID: c541bc66abe24435a6de3fe321d6c83c
I20260808 17:49:36.794209 28867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:36.799665 28867 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:36.800395 28867 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:36.804570 28867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ed3d0f3a724499db3d2b1e837ee16fd: Generated new TSK 0
I20260808 17:49:36.804764 28867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:36.812139 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 28739
W20260808 17:49:36.824800 28720 connection.cc:535] client connection to 127.0.0.1:36279 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36279): Connection reset by peer (error 104)
2026-08-08T17:49:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1555 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:36 dist-test-slave-tk9z krb5kdc[28884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:36 dist-test-slave-tk9z krb5kdc[28884](info): set up 2 sockets
Aug 08 17:49:36 dist-test-slave-tk9z krb5kdc[28884](info): commencing operation
krb5kdc: starting...
W20260808 17:49:38.054273 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:38 dist-test-slave-tk9z krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:38.179607 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:38.264971 28902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:38.265228 28902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:38.265269 28902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:38.266455 28902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:38.266526 28902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:38.266551 28902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:38.266572 28902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:38.266601 28902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:38.268085 28902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42133
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:38.268374 28902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:38.268621 28902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:38.273701 28913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:38.273706 28910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:38.273712 28909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:38.273979 28902 server_base.cc:1034] running on GCE node
I20260808 17:49:38.274235 28902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:38.274613 28902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:38.275781 28902 hybrid_clock.cc:648] HybridClock initialized: now 1786211378275772 us; error 48 us; skew 500 ppm
Aug 08 17:49:38 dist-test-slave-tk9z krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211378, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:38.278468 28902 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:38.278965 28902 webserver.cc:460] Webserver started at http://127.0.0.1:43251/ using document root <none> and password file <none>
I20260808 17:49:38.279213 28902 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:38.279281 28902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:38.279451 28902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:38.280560 28902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "1fc4720913b14765a645f27e25415cb5"
format_stamp: "Formatted at 2026-08-08 17:49:38 on dist-test-slave-tk9z"
server_key: "22d1554eebc547b57c6accdbe5c63c96"
server_key_iv: "d3de73dfbfa520154416e35dcdda7a1f"
server_key_version: "encryptionkey@0"
I20260808 17:49:38.280956 28902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "1fc4720913b14765a645f27e25415cb5"
format_stamp: "Formatted at 2026-08-08 17:49:38 on dist-test-slave-tk9z"
server_key: "22d1554eebc547b57c6accdbe5c63c96"
server_key_iv: "d3de73dfbfa520154416e35dcdda7a1f"
server_key_version: "encryptionkey@0"
I20260808 17:49:38.282346 28902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:38.283217 28921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:38.283386 28902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:38.283512 28902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "1fc4720913b14765a645f27e25415cb5"
format_stamp: "Formatted at 2026-08-08 17:49:38 on dist-test-slave-tk9z"
server_key: "22d1554eebc547b57c6accdbe5c63c96"
server_key_iv: "d3de73dfbfa520154416e35dcdda7a1f"
server_key_version: "encryptionkey@0"
I20260808 17:49:38.283598 28902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:38.287031 28902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:38.288918 28902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:38.289078 28902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:38.293203 28902 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42133
I20260808 17:49:38.293241 28999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42133 every 8 connection(s)
I20260808 17:49:38.293591 28902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:38.294688 29000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:38.295393 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 28902
I20260808 17:49:38.295528 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:38.295723 29947 external_mini_cluster.cc:1428] Setting key 08fb7f64c1ef6d9f5640e6f1cfec16bc
I20260808 17:49:38.298087 29000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5: Bootstrap starting.
I20260808 17:49:38.298309 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:38.296239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33983 (local address 127.0.0.1:42133)
0808 17:49:38.296426 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:38.296433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:38.296458 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:49:38.296860 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:38.296866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:38.296890 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:38.297010 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:38.297776 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:38.297780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:38.297781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:38.297785 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:38.297847 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:38.297863 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:38.297946 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:38.298022 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 17:49:38.298077 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260808 17:49:38.298935 29000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:38.299263 29000 log.cc:824] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:38.299425 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:38.300004 29000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5: No bootstrap required, opened a new log
I20260808 17:49:38.301865 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:38.300918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43473 (local address 127.0.0.1:42133)
0808 17:49:38.300955 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:38.300957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:38.301024 (+ 67us) server_negotiation.cc:407] Connection header received
0808 17:49:38.301090 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:38.301090 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:38.301094 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:38.301139 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:38.301655 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:38.301656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:38.301656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:38.301660 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:38.301672 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:38.301675 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:38.301718 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:38.301774 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 17:49:38.301784 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:49:38.302554 29000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc4720913b14765a645f27e25415cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42133 } }
I20260808 17:49:38.302714 29000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:38.302762 29000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc4720913b14765a645f27e25415cb5, State: Initialized, Role: FOLLOWER
I20260808 17:49:38.302883 29000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc4720913b14765a645f27e25415cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42133 } }
I20260808 17:49:38.303002 29000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:38.303049 29000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:38.303102 29000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:38.303642 29000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc4720913b14765a645f27e25415cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42133 } }
I20260808 17:49:38.303769 29000 leader_election.cc:302] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fc4720913b14765a645f27e25415cb5; no voters:
I20260808 17:49:38.303954 29000 leader_election.cc:288] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:38.303987 29014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:38.304167 29014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [term 1 LEADER]: Becoming Leader. State: Replica: 1fc4720913b14765a645f27e25415cb5, State: Running, Role: LEADER
I20260808 17:49:38.304209 29000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:38.304426 29014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc4720913b14765a645f27e25415cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42133 } }
I20260808 17:49:38.304804 29015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fc4720913b14765a645f27e25415cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc4720913b14765a645f27e25415cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42133 } } }
I20260808 17:49:38.304814 29017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fc4720913b14765a645f27e25415cb5. Latest consensus state: current_term: 1 leader_uuid: "1fc4720913b14765a645f27e25415cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc4720913b14765a645f27e25415cb5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42133 } } }
I20260808 17:49:38.304994 29017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:38.305210 29015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:38.305389 29030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:38.305948 29030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:38.307539 29030 catalog_manager.cc:1347] Generated new cluster ID: 63169587f3374dc2bdb57a10f9fa456a
I20260808 17:49:38.307580 29030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:38.342556 29030 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:38.342764 29030 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:38.346032 29030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fc4720913b14765a645f27e25415cb5: Generated new TSK 0
I20260808 17:49:38.346208 29030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:38.350534 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 28902
2026-08-08T17:49:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:38 dist-test-slave-tk9z krb5kdc[29056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:38 dist-test-slave-tk9z krb5kdc[29056](info): set up 2 sockets
Aug 08 17:49:38 dist-test-slave-tk9z krb5kdc[29056](info): commencing operation
krb5kdc: starting...
W20260808 17:49:39.604513 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:39 dist-test-slave-tk9z krb5kdc[29056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:39.724681 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:39.807909 29078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:39.808116 29078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:39.808139 29078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:39.809314 29078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:39.809360 29078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:39.809381 29078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:39.809394 29078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:39.809406 29078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:39.810959 29078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:39.811228 29078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:39.811480 29078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:39.816565 29083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:39.816617 29078 server_base.cc:1034] running on GCE node
W20260808 17:49:39.816582 29089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:39.816582 29084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:39.817008 29078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:39.817303 29078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:39.818437 29078 hybrid_clock.cc:648] HybridClock initialized: now 1786211379818434 us; error 39 us; skew 500 ppm
Aug 08 17:49:39 dist-test-slave-tk9z krb5kdc[29056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211379, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:39.820945 29078 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:39.821408 29078 webserver.cc:460] Webserver started at http://127.0.0.1:36625/ using document root <none> and password file <none>
I20260808 17:49:39.821658 29078 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:39.821727 29078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:39.821878 29078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:39.823062 29078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "b69e6d08164e4a33bb9a6457b6bc3842"
format_stamp: "Formatted at 2026-08-08 17:49:39 on dist-test-slave-tk9z"
server_key: "9e47c258917cb1205404edd14da9d0bb"
server_key_iv: "ad95dc7d80e33c5c15091b58222fd2a3"
server_key_version: "encryptionkey@0"
I20260808 17:49:39.823431 29078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "b69e6d08164e4a33bb9a6457b6bc3842"
format_stamp: "Formatted at 2026-08-08 17:49:39 on dist-test-slave-tk9z"
server_key: "9e47c258917cb1205404edd14da9d0bb"
server_key_iv: "ad95dc7d80e33c5c15091b58222fd2a3"
server_key_version: "encryptionkey@0"
I20260808 17:49:39.824812 29078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:49:39.825560 29099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:39.825745 29078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:39.825857 29078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "b69e6d08164e4a33bb9a6457b6bc3842"
format_stamp: "Formatted at 2026-08-08 17:49:39 on dist-test-slave-tk9z"
server_key: "9e47c258917cb1205404edd14da9d0bb"
server_key_iv: "ad95dc7d80e33c5c15091b58222fd2a3"
server_key_version: "encryptionkey@0"
I20260808 17:49:39.825943 29078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:39.840816 29078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:39.843286 29078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:39.843463 29078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:39.847818 29078 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33201
I20260808 17:49:39.847855 29172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33201 every 8 connection(s)
I20260808 17:49:39.848248 29078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:39.849177 29173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:39.850131 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 29078
I20260808 17:49:39.850281 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:39.850522 29947 external_mini_cluster.cc:1428] Setting key b46de872bb569b0a7e2ec7fb6783fa91
I20260808 17:49:39.852828 29173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842: Bootstrap starting.
I20260808 17:49:39.853485 29173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:39.853799 29173 log.cc:824] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:39.854624 29173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842: No bootstrap required, opened a new log
I20260808 17:49:39.857276 29173 raft_consensus.cc:348] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69e6d08164e4a33bb9a6457b6bc3842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33201 } }
I20260808 17:49:39.857422 29173 raft_consensus.cc:374] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:39.857445 29173 raft_consensus.cc:729] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69e6d08164e4a33bb9a6457b6bc3842, State: Initialized, Role: FOLLOWER
I20260808 17:49:39.857583 29173 consensus_queue.cc:260] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69e6d08164e4a33bb9a6457b6bc3842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33201 } }
I20260808 17:49:39.857653 29173 raft_consensus.cc:388] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:39.857676 29173 raft_consensus.cc:482] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:39.857750 29173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 17:49:39 dist-test-slave-tk9z krb5kdc[29056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:49:39.858421 29173 raft_consensus.cc:504] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69e6d08164e4a33bb9a6457b6bc3842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33201 } }
I20260808 17:49:39.858546 29173 leader_election.cc:302] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b69e6d08164e4a33bb9a6457b6bc3842; no voters:
I20260808 17:49:39.858717 29173 leader_election.cc:288] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:39.858750 29180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:39.858917 29180 raft_consensus.cc:686] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [term 1 LEADER]: Becoming Leader. State: Replica: b69e6d08164e4a33bb9a6457b6bc3842, State: Running, Role: LEADER
I20260808 17:49:39.859084 29173 sys_catalog.cc:564] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:39.859056 29180 consensus_queue.cc:237] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69e6d08164e4a33bb9a6457b6bc3842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33201 } }
I20260808 17:49:39.859534 29181 sys_catalog.cc:455] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b69e6d08164e4a33bb9a6457b6bc3842. Latest consensus state: current_term: 1 leader_uuid: "b69e6d08164e4a33bb9a6457b6bc3842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69e6d08164e4a33bb9a6457b6bc3842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33201 } } }
I20260808 17:49:39.859717 29182 sys_catalog.cc:455] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b69e6d08164e4a33bb9a6457b6bc3842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69e6d08164e4a33bb9a6457b6bc3842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33201 } } }
I20260808 17:49:39.860924 29182 sys_catalog.cc:458] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:39.860699 29181 sys_catalog.cc:458] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842 [sys.catalog]: This master's current role is: LEADER
W20260808 17:49:39.861136 29205 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:49:39.861225 29205 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:49:39.861299 29207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:39.861543 29207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:39.862564 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:39.851126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41581 (local address 127.0.0.1:33201)
0808 17:49:39.851334 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:39.851342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:39.851369 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:49:39.851811 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:39.851818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:39.851865 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:39.851983 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:39.852862 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:39.853901 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:39.854921 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:39.855311 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:39.855390 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:39.858411 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:39.858415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:39.858416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:39.858430 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:39.860666 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:39.861128 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:39.861131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:39.861132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:39.861276 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:39.861533 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:39.861534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:39.861534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:39.861896 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:39.862058 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:39.862182 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 17:49:39.862339 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
I20260808 17:49:39.863427 29207 catalog_manager.cc:1347] Generated new cluster ID: dd85b2a140b8414fbedb6d57076b14b4
I20260808 17:49:39.863511 29207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:39.863636 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:39.873898 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:39.866673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44997 (local address 127.0.0.1:33201)
0808 17:49:39.866712 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:39.866714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:39.866845 (+ 131us) server_negotiation.cc:407] Connection header received
0808 17:49:39.866909 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:39.866910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:39.866921 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:39.866974 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:39.867580 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:39.868352 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:39.869322 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:39.869660 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:39.869735 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:39.871069 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:39.871071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:39.871072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:39.871078 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:39.872682 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:39.873077 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:39.873078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:39.873079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:39.873133 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:39.873326 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:39.873327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:39.873327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:39.873444 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:39.873524 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:39.873727 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 17:49:39.873804 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:49:39.877255 29207 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:39.877964 29207 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:39.887797 29207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b69e6d08164e4a33bb9a6457b6bc3842: Generated new TSK 0
I20260808 17:49:39.888007 29207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:39.892076 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 29078
2026-08-08T17:49:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1551 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:39 dist-test-slave-tk9z krb5kdc[29217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:39 dist-test-slave-tk9z krb5kdc[29217](info): set up 2 sockets
Aug 08 17:49:39 dist-test-slave-tk9z krb5kdc[29217](info): commencing operation
krb5kdc: starting...
W20260808 17:49:41.167933 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:41 dist-test-slave-tk9z krb5kdc[29217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:41.293329 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:41.382907 29237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:41.383134 29237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:41.383160 29237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:41.384274 29237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:41.384315 29237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:41.384330 29237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:41.384341 29237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:41.384354 29237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:41.385814 29237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:41.386019 29237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:41.386189 29237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:41.391214 29246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:41.391182 29249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:41.391501 29237 server_base.cc:1034] running on GCE node
W20260808 17:49:41.391182 29247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:41.391779 29237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:41.392095 29237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:41.393270 29237 hybrid_clock.cc:648] HybridClock initialized: now 1786211381393248 us; error 38 us; skew 500 ppm
Aug 08 17:49:41 dist-test-slave-tk9z krb5kdc[29217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:41.395952 29237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:41.396378 29237 webserver.cc:460] Webserver started at http://127.0.0.1:42321/ using document root <none> and password file <none>
I20260808 17:49:41.396618 29237 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:41.396667 29237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:41.396802 29237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:41.397892 29237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "36fd9177b12d46828a5a72d4b89d3ef9"
format_stamp: "Formatted at 2026-08-08 17:49:41 on dist-test-slave-tk9z"
server_key: "272c6eda90b14564baf48ad2549c9710"
server_key_iv: "5bbcab6a7f75f617b4b4b3e434c9b0c0"
server_key_version: "encryptionkey@0"
I20260808 17:49:41.398276 29237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "36fd9177b12d46828a5a72d4b89d3ef9"
format_stamp: "Formatted at 2026-08-08 17:49:41 on dist-test-slave-tk9z"
server_key: "272c6eda90b14564baf48ad2549c9710"
server_key_iv: "5bbcab6a7f75f617b4b4b3e434c9b0c0"
server_key_version: "encryptionkey@0"
I20260808 17:49:41.399771 29237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:41.400502 29257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:41.400698 29237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:41.400804 29237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "36fd9177b12d46828a5a72d4b89d3ef9"
format_stamp: "Formatted at 2026-08-08 17:49:41 on dist-test-slave-tk9z"
server_key: "272c6eda90b14564baf48ad2549c9710"
server_key_iv: "5bbcab6a7f75f617b4b4b3e434c9b0c0"
server_key_version: "encryptionkey@0"
I20260808 17:49:41.400876 29237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:41.408934 29237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:41.411046 29237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:41.411186 29237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:41.415093 29237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41555
I20260808 17:49:41.415143 29333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41555 every 8 connection(s)
I20260808 17:49:41.415518 29237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:41.416431 29334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:41.418794 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 29237
I20260808 17:49:41.418802 29334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9: Bootstrap starting.
I20260808 17:49:41.418915 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:41.419109 29947 external_mini_cluster.cc:1428] Setting key 0d0644f0ba9b6f4e90dea0f87eb6bd3a
I20260808 17:49:41.419548 29334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:41.419929 29334 log.cc:824] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:41.420670 29334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9: No bootstrap required, opened a new log
I20260808 17:49:41.422794 29334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd9177b12d46828a5a72d4b89d3ef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260808 17:49:41.422968 29334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:41.423076 29334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fd9177b12d46828a5a72d4b89d3ef9, State: Initialized, Role: FOLLOWER
I20260808 17:49:41.423228 29334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd9177b12d46828a5a72d4b89d3ef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260808 17:49:41.423313 29334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:41.423355 29334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:41.423405 29334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:41.424083 29334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd9177b12d46828a5a72d4b89d3ef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260808 17:49:41.424206 29334 leader_election.cc:302] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36fd9177b12d46828a5a72d4b89d3ef9; no voters:
I20260808 17:49:41.424371 29334 leader_election.cc:288] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:41.424394 29340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:41.424539 29340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [term 1 LEADER]: Becoming Leader. State: Replica: 36fd9177b12d46828a5a72d4b89d3ef9, State: Running, Role: LEADER
I20260808 17:49:41.424667 29334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:41.424697 29340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd9177b12d46828a5a72d4b89d3ef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260808 17:49:41.425201 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:41.419643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46913 (local address 127.0.0.1:41555)
0808 17:49:41.419832 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:41.419839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:41.419864 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:49:41.420501 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:41.420509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:41.420558 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:41.420672 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:41.420929 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:41.421798 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:41.423040 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:41.423640 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:41.423722 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:41.424422 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:41.424426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:41.424426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:41.424647 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:41.424690 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:41.424705 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:41.424937 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:41.424967 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 17:49:41.425029 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260808 17:49:41.425290 29343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36fd9177b12d46828a5a72d4b89d3ef9. Latest consensus state: current_term: 1 leader_uuid: "36fd9177b12d46828a5a72d4b89d3ef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd9177b12d46828a5a72d4b89d3ef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } } }
I20260808 17:49:41.425393 29343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:41.426008 29341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36fd9177b12d46828a5a72d4b89d3ef9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fd9177b12d46828a5a72d4b89d3ef9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } } }
I20260808 17:49:41.426456 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:41.426514 29341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9 [sys.catalog]: This master's current role is: LEADER
W20260808 17:49:41.426563 29365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:49:41.426600 29365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:49:41.426669 29354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:41.426847 29354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:41.428447 29354 catalog_manager.cc:1347] Generated new cluster ID: 00e68e0fa9334e2298170ea6b617de5a
I20260808 17:49:41.428503 29354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:41.430816 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:41.427713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56937 (local address 127.0.0.1:41555)
0808 17:49:41.427775 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:41.427777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:41.427993 (+ 216us) server_negotiation.cc:407] Connection header received
0808 17:49:41.428066 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:41.428067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:41.428078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:41.428127 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:41.428299 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:41.428860 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:41.429593 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:41.429800 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:41.429887 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:41.430575 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:41.430576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:41.430576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:41.430581 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:41.430593 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:41.430597 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:41.430642 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:41.430707 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:49:41.430726 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:49:41.455600 29354 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:41.456157 29354 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:41.468772 29354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36fd9177b12d46828a5a72d4b89d3ef9: Generated new TSK 0
I20260808 17:49:41.469005 29354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:41.484786 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 29237
2026-08-08T17:49:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:41 dist-test-slave-tk9z krb5kdc[29377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:41 dist-test-slave-tk9z krb5kdc[29377](info): set up 2 sockets
Aug 08 17:49:41 dist-test-slave-tk9z krb5kdc[29377](info): commencing operation
krb5kdc: starting...
W20260808 17:49:42.732975 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:42 dist-test-slave-tk9z krb5kdc[29377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:42.858702 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:42.946254 29398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:42.946535 29398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:42.946581 29398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:42.947750 29398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:42.947818 29398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:42.947844 29398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:42.947871 29398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:42.947901 29398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:42.949378 29398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:42.949669 29398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:42.949904 29398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:42.954865 29412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:42.955034 29398 server_base.cc:1034] running on GCE node
W20260808 17:49:42.954912 29409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:42.954861 29408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:42.955384 29398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:42.955744 29398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:42.956897 29398 hybrid_clock.cc:648] HybridClock initialized: now 1786211382956874 us; error 39 us; skew 500 ppm
Aug 08 17:49:42 dist-test-slave-tk9z krb5kdc[29377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211382, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:42.963873 29398 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:42.964432 29398 webserver.cc:460] Webserver started at http://127.0.0.1:43379/ using document root <none> and password file <none>
I20260808 17:49:42.964709 29398 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:42.964781 29398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:42.964936 29398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:42.966038 29398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "e2449e36f4b049c6b318f7655b985f4d"
format_stamp: "Formatted at 2026-08-08 17:49:42 on dist-test-slave-tk9z"
server_key: "b106fc9ea404be15a66eda52a605a8ac"
server_key_iv: "c51aa4a281ba85e2581f0d4e3a2ca802"
server_key_version: "encryptionkey@0"
I20260808 17:49:42.966497 29398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "e2449e36f4b049c6b318f7655b985f4d"
format_stamp: "Formatted at 2026-08-08 17:49:42 on dist-test-slave-tk9z"
server_key: "b106fc9ea404be15a66eda52a605a8ac"
server_key_iv: "c51aa4a281ba85e2581f0d4e3a2ca802"
server_key_version: "encryptionkey@0"
I20260808 17:49:42.967895 29398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:42.968693 29419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:42.968918 29398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:42.969029 29398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "e2449e36f4b049c6b318f7655b985f4d"
format_stamp: "Formatted at 2026-08-08 17:49:42 on dist-test-slave-tk9z"
server_key: "b106fc9ea404be15a66eda52a605a8ac"
server_key_iv: "c51aa4a281ba85e2581f0d4e3a2ca802"
server_key_version: "encryptionkey@0"
I20260808 17:49:42.969121 29398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:42.972525 29398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:42.974427 29398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:42.974596 29398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:42.978574 29398 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39765
I20260808 17:49:42.978605 29488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39765 every 8 connection(s)
I20260808 17:49:42.978963 29398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:42.980020 29490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:42.982333 29490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d: Bootstrap starting.
I20260808 17:49:42.982970 29490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:42.983212 29490 log.cc:824] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:42.983904 29490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d: No bootstrap required, opened a new log
I20260808 17:49:42.983983 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 29398
I20260808 17:49:42.984123 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:42.984333 29947 external_mini_cluster.cc:1428] Setting key 9b2cd6b48e2e943f8c44f0788c2f8286
I20260808 17:49:42.986009 29490 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2449e36f4b049c6b318f7655b985f4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39765 } }
I20260808 17:49:42.986186 29490 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:42.986225 29490 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2449e36f4b049c6b318f7655b985f4d, State: Initialized, Role: FOLLOWER
I20260808 17:49:42.986459 29490 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2449e36f4b049c6b318f7655b985f4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39765 } }
I20260808 17:49:42.986577 29490 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:42.986611 29490 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:42.986660 29490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:42.987452 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:42.984935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59717 (local address 127.0.0.1:39765)
0808 17:49:42.985106 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:42.985115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:42.985149 (+ 34us) server_negotiation.cc:407] Connection header received
0808 17:49:42.985995 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:42.986003 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:42.986039 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:42.986168 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:42.986940 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:42.986944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:42.986944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:42.986948 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:42.986995 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:42.987007 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:42.987068 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:42.987167 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 17:49:42.987199 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260808 17:49:42.987793 29490 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2449e36f4b049c6b318f7655b985f4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39765 } }
I20260808 17:49:42.987906 29490 leader_election.cc:302] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2449e36f4b049c6b318f7655b985f4d; no voters:
I20260808 17:49:42.988086 29490 leader_election.cc:288] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:42.988126 29495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:42.988293 29495 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [term 1 LEADER]: Becoming Leader. State: Replica: e2449e36f4b049c6b318f7655b985f4d, State: Running, Role: LEADER
I20260808 17:49:42.988353 29490 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:42.988437 29495 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2449e36f4b049c6b318f7655b985f4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39765 } }
I20260808 17:49:42.988615 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:42.988881 29495 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2449e36f4b049c6b318f7655b985f4d. Latest consensus state: current_term: 1 leader_uuid: "e2449e36f4b049c6b318f7655b985f4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2449e36f4b049c6b318f7655b985f4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39765 } } }
I20260808 17:49:42.988945 29495 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:42.988996 29497 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2449e36f4b049c6b318f7655b985f4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2449e36f4b049c6b318f7655b985f4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39765 } } }
I20260808 17:49:42.989086 29497 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:42.989850 29507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:42.990506 29507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:42.991480 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:42.990233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34319 (local address 127.0.0.1:39765)
0808 17:49:42.990279 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:42.990281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:42.990491 (+ 210us) server_negotiation.cc:407] Connection header received
0808 17:49:42.990578 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:42.990579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:42.990588 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:42.990641 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:42.991257 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:42.991258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:42.991259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:42.991263 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:42.991277 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:42.991281 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:42.991331 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:42.991386 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 17:49:42.991401 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:49:42.992184 29507 catalog_manager.cc:1347] Generated new cluster ID: fd5b30564abb4fb28858349ef4ec5aee
I20260808 17:49:42.992234 29507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:43.003078 29507 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:43.003278 29507 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:43.009200 29507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2449e36f4b049c6b318f7655b985f4d: Generated new TSK 0
I20260808 17:49:43.009403 29507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:43.014122 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 29398
2026-08-08T17:49:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:43 dist-test-slave-tk9z krb5kdc[29532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:43 dist-test-slave-tk9z krb5kdc[29532](info): set up 2 sockets
Aug 08 17:49:43 dist-test-slave-tk9z krb5kdc[29532](info): commencing operation
krb5kdc: starting...
W20260808 17:49:44.263793 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:44 dist-test-slave-tk9z krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:44.383339 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42927
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:44.471696 29547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:44.471921 29547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:44.471944 29547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:44.473049 29547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:44.473085 29547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:44.473100 29547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:44.473114 29547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:44.473125 29547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:44.474594 29547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:44.474790 29547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:44.474959 29547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:44.479557 29557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:44.479689 29547 server_base.cc:1034] running on GCE node
W20260808 17:49:44.479784 29561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:44.479558 29558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:44.480057 29547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:44.480361 29547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:44.481508 29547 hybrid_clock.cc:648] HybridClock initialized: now 1786211384481495 us; error 42 us; skew 500 ppm
Aug 08 17:49:44 dist-test-slave-tk9z krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211384, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:44.483956 29547 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:44.484361 29547 webserver.cc:460] Webserver started at http://127.0.0.1:39975/ using document root <none> and password file <none>
I20260808 17:49:44.484581 29547 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:44.484644 29547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:44.484782 29547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:44.485919 29547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "31dd15bfbf564255a22be67090dcba89"
format_stamp: "Formatted at 2026-08-08 17:49:44 on dist-test-slave-tk9z"
server_key: "9f89317e91f811e9f9fccf65a254eee6"
server_key_iv: "a88da983b486649b94a90a16a102b689"
server_key_version: "encryptionkey@0"
I20260808 17:49:44.486287 29547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "31dd15bfbf564255a22be67090dcba89"
format_stamp: "Formatted at 2026-08-08 17:49:44 on dist-test-slave-tk9z"
server_key: "9f89317e91f811e9f9fccf65a254eee6"
server_key_iv: "a88da983b486649b94a90a16a102b689"
server_key_version: "encryptionkey@0"
I20260808 17:49:44.487625 29547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:44.488303 29570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:44.488464 29547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:44.488564 29547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "31dd15bfbf564255a22be67090dcba89"
format_stamp: "Formatted at 2026-08-08 17:49:44 on dist-test-slave-tk9z"
server_key: "9f89317e91f811e9f9fccf65a254eee6"
server_key_iv: "a88da983b486649b94a90a16a102b689"
server_key_version: "encryptionkey@0"
I20260808 17:49:44.488641 29547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:44.507422 29547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:44.509702 29547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:44.509863 29547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:44.513772 29547 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42927
I20260808 17:49:44.513808 29644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42927 every 8 connection(s)
I20260808 17:49:44.514173 29547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:44.515080 29645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:44.517401 29645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89: Bootstrap starting.
I20260808 17:49:44.517951 29645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:44.518190 29645 log.cc:824] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:44.518736 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 29547
I20260808 17:49:44.518874 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:44.519017 29645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89: No bootstrap required, opened a new log
I20260808 17:49:44.519106 29947 external_mini_cluster.cc:1428] Setting key b5a31b54bbd23bc3d3d6e54f887ec4cc
I20260808 17:49:44.521428 29645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31dd15bfbf564255a22be67090dcba89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } }
I20260808 17:49:44.521577 29645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:44.521601 29645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31dd15bfbf564255a22be67090dcba89, State: Initialized, Role: FOLLOWER
I20260808 17:49:44.521734 29645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31dd15bfbf564255a22be67090dcba89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } }
I20260808 17:49:44.521797 29645 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:44.521847 29645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:44.521912 29645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:44.522533 29645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31dd15bfbf564255a22be67090dcba89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } }
I20260808 17:49:44.522645 29645 leader_election.cc:302] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31dd15bfbf564255a22be67090dcba89; no voters:
I20260808 17:49:44.522830 29645 leader_election.cc:288] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:44.522861 29652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:44.523032 29652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [term 1 LEADER]: Becoming Leader. State: Replica: 31dd15bfbf564255a22be67090dcba89, State: Running, Role: LEADER
I20260808 17:49:44.523169 29645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:44.523181 29652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31dd15bfbf564255a22be67090dcba89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } }
I20260808 17:49:44.523659 29655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31dd15bfbf564255a22be67090dcba89. Latest consensus state: current_term: 1 leader_uuid: "31dd15bfbf564255a22be67090dcba89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31dd15bfbf564255a22be67090dcba89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } } }
I20260808 17:49:44.523758 29655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:44.523998 29653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31dd15bfbf564255a22be67090dcba89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31dd15bfbf564255a22be67090dcba89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42927 } } }
I20260808 17:49:44.524269 29653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:44.524462 29664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:44.524989 29664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:49:44 dist-test-slave-tk9z krb5kdc[29532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:49:44.527208 29664 catalog_manager.cc:1347] Generated new cluster ID: d8ba570ac10f495ab6e385ed582e72c3
I20260808 17:49:44.527263 29664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:44.530861 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:44.519731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:47397 (local address 127.0.0.1:42927)
0808 17:49:44.519915 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:44.519928 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:44.519954 (+ 26us) server_negotiation.cc:407] Connection header received
0808 17:49:44.520357 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:44.520363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:44.520411 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:44.520523 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:44.521364 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:44.522810 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:44.523768 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:44.524395 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:44.524507 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:44.527607 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:44.527611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:44.527611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:44.527623 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:44.529292 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:44.529681 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:44.529684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:44.529685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:44.529774 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:44.529997 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:44.529998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:44.529999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:44.530226 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:44.530443 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:44.530569 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 17:49:44.530674 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260808 17:49:44.531880 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:44.539904 29664 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:44.540469 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:44.533424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:33183 (local address 127.0.0.1:42927)
0808 17:49:44.533461 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:44.533463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:44.534546 (+ 1083us) server_negotiation.cc:407] Connection header received
0808 17:49:44.534616 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:44.534617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:44.534627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:44.534690 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:44.535176 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:44.535660 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:44.536333 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:44.536547 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:44.536617 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:44.537732 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:44.537733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:44.537734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:44.537740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:44.539080 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:44.539450 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:44.539451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:44.539451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:44.539522 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:44.539800 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:44.539801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:44.539801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:44.540039 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:44.540153 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:44.540274 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 17:49:44.540336 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:49:44.540606 29664 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:44.554957 29664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31dd15bfbf564255a22be67090dcba89: Generated new TSK 0
I20260808 17:49:44.555171 29664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:44.558892 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 29547
W20260808 17:49:44.571359 29529 connection.cc:535] client connection to 127.0.0.1:42927 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42927): Connection reset by peer (error 104)
2026-08-08T17:49:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:44 dist-test-slave-tk9z krb5kdc[29694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:44 dist-test-slave-tk9z krb5kdc[29694](info): set up 2 sockets
Aug 08 17:49:44 dist-test-slave-tk9z krb5kdc[29694](info): commencing operation
krb5kdc: starting...
W20260808 17:49:45.786056 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:45 dist-test-slave-tk9z krb5kdc[29694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:45.909188 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33685
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:45.994961 29713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:45.995159 29713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:45.995182 29713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:45.996308 29713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:45.996351 29713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:45.996366 29713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:45.996415 29713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:45.996434 29713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:45.997892 29713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:45.998133 29713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:45.998360 29713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:46.003197 29721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:46.003209 29719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:46.003257 29713 server_base.cc:1034] running on GCE node
W20260808 17:49:46.003211 29718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:46.003701 29713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:46.003953 29713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:46.005115 29713 hybrid_clock.cc:648] HybridClock initialized: now 1786211386005096 us; error 44 us; skew 500 ppm
Aug 08 17:49:46 dist-test-slave-tk9z krb5kdc[29694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:46.007730 29713 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:46.008116 29713 webserver.cc:460] Webserver started at http://127.0.0.1:42825/ using document root <none> and password file <none>
I20260808 17:49:46.008296 29713 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:46.008335 29713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:46.008437 29713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:46.009509 29713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "d3457c06bd914ec796570a8b49f35728"
format_stamp: "Formatted at 2026-08-08 17:49:46 on dist-test-slave-tk9z"
server_key: "001ff8b4eb53e4dd0edec6b632f557a1"
server_key_iv: "15e08bd42dded5e43df438c8b302cf45"
server_key_version: "encryptionkey@0"
I20260808 17:49:46.009900 29713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "d3457c06bd914ec796570a8b49f35728"
format_stamp: "Formatted at 2026-08-08 17:49:46 on dist-test-slave-tk9z"
server_key: "001ff8b4eb53e4dd0edec6b632f557a1"
server_key_iv: "15e08bd42dded5e43df438c8b302cf45"
server_key_version: "encryptionkey@0"
I20260808 17:49:46.011348 29713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:46.012082 29741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:46.012244 29713 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:46.012349 29713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "d3457c06bd914ec796570a8b49f35728"
format_stamp: "Formatted at 2026-08-08 17:49:46 on dist-test-slave-tk9z"
server_key: "001ff8b4eb53e4dd0edec6b632f557a1"
server_key_iv: "15e08bd42dded5e43df438c8b302cf45"
server_key_version: "encryptionkey@0"
I20260808 17:49:46.012435 29713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:46.027078 29713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:46.029294 29713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:46.029453 29713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:46.033272 29713 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33685
I20260808 17:49:46.033309 29824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33685 every 8 connection(s)
I20260808 17:49:46.033661 29713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:46.034607 29827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:46.035194 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 29713
I20260808 17:49:46.035326 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:46.035516 29947 external_mini_cluster.cc:1428] Setting key 2a35d29ec179cef724f4ec9c18df7d8b
I20260808 17:49:46.037601 29827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728: Bootstrap starting.
I20260808 17:49:46.038257 29827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:46.038616 29827 log.cc:824] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:46.039348 29827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728: No bootstrap required, opened a new log
I20260808 17:49:46.040987 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:46.036056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:56977 (local address 127.0.0.1:33685)
0808 17:49:46.036226 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:46.036233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:46.036258 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:49:46.036670 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:46.036676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:46.036723 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:46.036835 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:46.037091 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:46.038019 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:46.039118 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:46.039566 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:46.039647 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:46.040310 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:46.040314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:46.040314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:46.040481 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:46.040510 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:46.040521 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:46.040584 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:46.040741 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 17:49:46.040774 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260808 17:49:46.041742 29827 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3457c06bd914ec796570a8b49f35728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } }
I20260808 17:49:46.041924 29827 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:46.041985 29827 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3457c06bd914ec796570a8b49f35728, State: Initialized, Role: FOLLOWER
I20260808 17:49:46.042089 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:46.042120 29827 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3457c06bd914ec796570a8b49f35728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } }
I20260808 17:49:46.042207 29827 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:46.042241 29827 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:46.042285 29827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:46.043040 29827 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3457c06bd914ec796570a8b49f35728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } }
I20260808 17:49:46.043154 29827 leader_election.cc:302] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3457c06bd914ec796570a8b49f35728; no voters:
I20260808 17:49:46.043337 29827 leader_election.cc:288] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:46.043426 29836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:46.043612 29827 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:46.043908 29836 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [term 1 LEADER]: Becoming Leader. State: Replica: d3457c06bd914ec796570a8b49f35728, State: Running, Role: LEADER
I20260808 17:49:46.044061 29836 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3457c06bd914ec796570a8b49f35728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } }
I20260808 17:49:46.044488 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3457c06bd914ec796570a8b49f35728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3457c06bd914ec796570a8b49f35728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } } }
I20260808 17:49:46.044574 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:46.044734 29840 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3457c06bd914ec796570a8b49f35728. Latest consensus state: current_term: 1 leader_uuid: "d3457c06bd914ec796570a8b49f35728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3457c06bd914ec796570a8b49f35728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33685 } } }
I20260808 17:49:46.044803 29840 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728 [sys.catalog]: This master's current role is: LEADER
W20260808 17:49:46.045559 29859 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:49:46.045616 29859 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:49:46.045667 29849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:46.045851 29849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:46.047286 29849 catalog_manager.cc:1347] Generated new cluster ID: ab3068648ef5484293cb0ec0245d3a41
I20260808 17:49:46.047358 29849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:46.047432 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:46.043734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:36995 (local address 127.0.0.1:33685)
0808 17:49:46.043770 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:46.043772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:46.044039 (+ 267us) server_negotiation.cc:407] Connection header received
0808 17:49:46.044117 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:46.044118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:46.044128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:46.044188 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:46.044366 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:46.045153 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:46.046072 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:46.046283 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:46.046384 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:46.047059 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:46.047060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:46.047060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:46.047064 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:46.047078 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:46.047083 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:46.047130 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:46.047282 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 17:49:46.047291 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:49:46.054184 29849 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:46.054749 29849 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:46.058773 29849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3457c06bd914ec796570a8b49f35728: Generated new TSK 0
I20260808 17:49:46.058945 29849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:46.066459 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 29713
W20260808 17:49:46.082574 29691 connection.cc:535] client connection to 127.0.0.1:33685 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33685): Connection reset by peer (error 104)
2026-08-08T17:49:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:46 dist-test-slave-tk9z krb5kdc[29867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:46 dist-test-slave-tk9z krb5kdc[29867](info): set up 2 sockets
Aug 08 17:49:46 dist-test-slave-tk9z krb5kdc[29867](info): commencing operation
krb5kdc: starting...
W20260808 17:49:47.296583 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:47 dist-test-slave-tk9z krb5kdc[29867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:47.418720 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:47.499712 29895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:47.499943 29895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:47.499984 29895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:47.501120 29895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:47.501184 29895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:47.501209 29895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:47.501240 29895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:47.501263 29895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:47.502826 29895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:47.503096 29895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:47.503314 29895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:47.508133 29905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:47.508186 29901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:47.508131 29900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:47.508206 29895 server_base.cc:1034] running on GCE node
I20260808 17:49:47.508592 29895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:47.508925 29895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:47.510084 29895 hybrid_clock.cc:648] HybridClock initialized: now 1786211387510069 us; error 44 us; skew 500 ppm
Aug 08 17:49:47 dist-test-slave-tk9z krb5kdc[29867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:47.512481 29895 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:47.512936 29895 webserver.cc:460] Webserver started at http://127.0.0.1:38809/ using document root <none> and password file <none>
I20260808 17:49:47.513166 29895 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:47.513229 29895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:47.513377 29895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:47.514472 29895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "33a4a0a0150948ba9794ded3a3d03e2c"
format_stamp: "Formatted at 2026-08-08 17:49:47 on dist-test-slave-tk9z"
server_key: "81da5f479f362199935c447d11b00a9a"
server_key_iv: "ff761b5924e8dee4b5446319d57dbfbf"
server_key_version: "encryptionkey@0"
I20260808 17:49:47.514827 29895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "33a4a0a0150948ba9794ded3a3d03e2c"
format_stamp: "Formatted at 2026-08-08 17:49:47 on dist-test-slave-tk9z"
server_key: "81da5f479f362199935c447d11b00a9a"
server_key_iv: "ff761b5924e8dee4b5446319d57dbfbf"
server_key_version: "encryptionkey@0"
I20260808 17:49:47.516171 29895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:47.516868 29913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:47.517030 29895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:47.517128 29895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "33a4a0a0150948ba9794ded3a3d03e2c"
format_stamp: "Formatted at 2026-08-08 17:49:47 on dist-test-slave-tk9z"
server_key: "81da5f479f362199935c447d11b00a9a"
server_key_iv: "ff761b5924e8dee4b5446319d57dbfbf"
server_key_version: "encryptionkey@0"
I20260808 17:49:47.517203 29895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:47.520471 29895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:47.522225 29895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:47.522413 29895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:47.526255 29895 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37935
I20260808 17:49:47.526289 30005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37935 every 8 connection(s)
I20260808 17:49:47.526666 29895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:47.527644 30007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:47.529870 30007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c: Bootstrap starting.
I20260808 17:49:47.530432 30007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:47.530676 30007 log.cc:824] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:47.531325 30007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c: No bootstrap required, opened a new log
I20260808 17:49:47.532876 30007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a4a0a0150948ba9794ded3a3d03e2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } }
I20260808 17:49:47.533016 30007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:47.533039 30007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a4a0a0150948ba9794ded3a3d03e2c, State: Initialized, Role: FOLLOWER
I20260808 17:49:47.533175 30007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a4a0a0150948ba9794ded3a3d03e2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } }
I20260808 17:49:47.533264 30007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:47.533322 30007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:47.533382 30007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:47.533488 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 29895
I20260808 17:49:47.533627 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:47.533820 29947 external_mini_cluster.cc:1428] Setting key abf0756db51c0bb3b9766e573b9a20b0
I20260808 17:49:47.534163 30007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a4a0a0150948ba9794ded3a3d03e2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } }
I20260808 17:49:47.534317 30007 leader_election.cc:302] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33a4a0a0150948ba9794ded3a3d03e2c; no voters:
I20260808 17:49:47.534523 30007 leader_election.cc:288] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:47.534610 30012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:47.534794 30012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [term 1 LEADER]: Becoming Leader. State: Replica: 33a4a0a0150948ba9794ded3a3d03e2c, State: Running, Role: LEADER
I20260808 17:49:47.534811 30007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:47.534919 30012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a4a0a0150948ba9794ded3a3d03e2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } }
I20260808 17:49:47.535689 30012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33a4a0a0150948ba9794ded3a3d03e2c. Latest consensus state: current_term: 1 leader_uuid: "33a4a0a0150948ba9794ded3a3d03e2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a4a0a0150948ba9794ded3a3d03e2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } } }
I20260808 17:49:47.536163 30012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:47.536232 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:47.534309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:50659 (local address 127.0.0.1:37935)
0808 17:49:47.534539 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:47.534548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:47.534598 (+ 50us) server_negotiation.cc:407] Connection header received
0808 17:49:47.534721 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:47.534729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:47.534757 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:47.534894 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:47.535632 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:47.535636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:47.535637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:47.535641 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:47.535694 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:47.535709 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:47.535945 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:47.535978 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 17:49:47.536022 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":138,"threads_started":1}
I20260808 17:49:47.536062 30014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33a4a0a0150948ba9794ded3a3d03e2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a4a0a0150948ba9794ded3a3d03e2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } } }
I20260808 17:49:47.536713 30014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c [sys.catalog]: This master's current role is: LEADER
W20260808 17:49:47.537230 30038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 17:49:47.537266 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 17:49:47.537286 30038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:49:47.537333 30026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:47.537492 30026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:47.539141 30026 catalog_manager.cc:1347] Generated new cluster ID: 31137b9b645143ee8a4b1472e7035310
I20260808 17:49:47.539201 30026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:47.540050 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:47.538825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:47749 (local address 127.0.0.1:37935)
0808 17:49:47.538860 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:47.538862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:47.539100 (+ 238us) server_negotiation.cc:407] Connection header received
0808 17:49:47.539151 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:47.539152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:47.539158 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:47.539211 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:47.539830 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:47.539831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:47.539831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:47.539835 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:47.539850 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:47.539854 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:47.539916 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:47.539981 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:49:47.539992 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:49:47.550783 30026 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:47.550976 30026 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:47.555920 30026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33a4a0a0150948ba9794ded3a3d03e2c: Generated new TSK 0
I20260808 17:49:47.556119 30026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:47.565078 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 29895
2026-08-08T17:49:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:47 dist-test-slave-tk9z krb5kdc[30051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:47 dist-test-slave-tk9z krb5kdc[30051](info): set up 2 sockets
Aug 08 17:49:47 dist-test-slave-tk9z krb5kdc[30051](info): commencing operation
krb5kdc: starting...
W20260808 17:49:48.816589 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:48 dist-test-slave-tk9z krb5kdc[30051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:48.938458 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41565
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:49.019721 30071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:49.019948 30071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:49.019987 30071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:49.021113 30071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:49.021174 30071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:49.021202 30071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:49.021239 30071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:49.021270 30071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:49.022809 30071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41565
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:49.023077 30071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:49.023288 30071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:49:49.028187 30071 server_base.cc:1034] running on GCE node
W20260808 17:49:49.028141 30079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:49.028154 30082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:49.028187 30080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:49.028703 30071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:49.029009 30071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:49.030166 30071 hybrid_clock.cc:648] HybridClock initialized: now 1786211389030151 us; error 27 us; skew 500 ppm
Aug 08 17:49:49 dist-test-slave-tk9z krb5kdc[30051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:49.037446 30071 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:49.037882 30071 webserver.cc:460] Webserver started at http://127.0.0.1:46757/ using document root <none> and password file <none>
I20260808 17:49:49.038128 30071 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:49.038194 30071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:49.038341 30071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:49.039420 30071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "c264b969f2514053bad36f10ab91aae4"
format_stamp: "Formatted at 2026-08-08 17:49:49 on dist-test-slave-tk9z"
server_key: "4be3dd2b5aff97b69cea47ba83c0ea5e"
server_key_iv: "88f9750c95a73469443e72a696ec544c"
server_key_version: "encryptionkey@0"
I20260808 17:49:49.039780 30071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "c264b969f2514053bad36f10ab91aae4"
format_stamp: "Formatted at 2026-08-08 17:49:49 on dist-test-slave-tk9z"
server_key: "4be3dd2b5aff97b69cea47ba83c0ea5e"
server_key_iv: "88f9750c95a73469443e72a696ec544c"
server_key_version: "encryptionkey@0"
I20260808 17:49:49.041169 30071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:49:49.042080 30097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:49.042290 30071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:49.042413 30071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "c264b969f2514053bad36f10ab91aae4"
format_stamp: "Formatted at 2026-08-08 17:49:49 on dist-test-slave-tk9z"
server_key: "4be3dd2b5aff97b69cea47ba83c0ea5e"
server_key_iv: "88f9750c95a73469443e72a696ec544c"
server_key_version: "encryptionkey@0"
I20260808 17:49:49.042496 30071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:49.054760 30071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:49.056767 30071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:49.056931 30071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:49.061009 30071 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41565
I20260808 17:49:49.061059 30174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41565 every 8 connection(s)
I20260808 17:49:49.061407 30071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:49.062292 30175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:49.063222 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30071
I20260808 17:49:49.063356 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:49.063531 29947 external_mini_cluster.cc:1428] Setting key 61c9f70170d5bd9cb6c06d90a9eac074
I20260808 17:49:49.065001 30175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4: Bootstrap starting.
I20260808 17:49:49.065764 30175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:49.066080 30175 log.cc:824] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:49.066836 30175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4: No bootstrap required, opened a new log
I20260808 17:49:49.068629 30175 raft_consensus.cc:348] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264b969f2514053bad36f10ab91aae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41565 } }
I20260808 17:49:49.068764 30175 raft_consensus.cc:374] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:49.068786 30175 raft_consensus.cc:729] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c264b969f2514053bad36f10ab91aae4, State: Initialized, Role: FOLLOWER
I20260808 17:49:49.068931 30175 consensus_queue.cc:260] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264b969f2514053bad36f10ab91aae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41565 } }
I20260808 17:49:49.068994 30175 raft_consensus.cc:388] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:49.069043 30175 raft_consensus.cc:482] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:49.069092 30175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:49.069628 30175 raft_consensus.cc:504] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264b969f2514053bad36f10ab91aae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41565 } }
I20260808 17:49:49.069749 30175 leader_election.cc:302] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c264b969f2514053bad36f10ab91aae4; no voters:
I20260808 17:49:49.069916 30175 leader_election.cc:288] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:49.069963 30185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:49.070165 30185 raft_consensus.cc:686] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [term 1 LEADER]: Becoming Leader. State: Replica: c264b969f2514053bad36f10ab91aae4, State: Running, Role: LEADER
I20260808 17:49:49.070173 30175 sys_catalog.cc:564] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:49.070283 30185 consensus_queue.cc:237] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264b969f2514053bad36f10ab91aae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41565 } }
Aug 08 17:49:49 dist-test-slave-tk9z krb5kdc[30051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:49:49.070761 30186 sys_catalog.cc:455] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c264b969f2514053bad36f10ab91aae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264b969f2514053bad36f10ab91aae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41565 } } }
I20260808 17:49:49.071019 30186 sys_catalog.cc:458] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:49.070937 30188 sys_catalog.cc:455] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c264b969f2514053bad36f10ab91aae4. Latest consensus state: current_term: 1 leader_uuid: "c264b969f2514053bad36f10ab91aae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264b969f2514053bad36f10ab91aae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41565 } } }
I20260808 17:49:49.071455 30195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:49.071310 30188 sys_catalog.cc:458] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:49.072046 30195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:49.073645 30195 catalog_manager.cc:1347] Generated new cluster ID: c209d67098e44c6aa92cc0eefcb2e5eb
I20260808 17:49:49.073700 30195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:49.075194 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:49.064082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:51721 (local address 127.0.0.1:41565)
0808 17:49:49.064249 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:49.064256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:49.064282 (+ 26us) server_negotiation.cc:407] Connection header received
0808 17:49:49.064824 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:49.064831 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:49.064879 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:49.064996 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:49.065843 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:49.066909 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:49.067673 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:49.068075 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:49.068190 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:49.071062 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:49.071066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:49.071067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:49.071080 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:49.073392 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:49.073840 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:49.073843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:49.073843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:49.073949 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:49.074190 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:49.074191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:49.074192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:49.074523 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:49.074666 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:49.074917 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 17:49:49.075021 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260808 17:49:49.079070 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:49.086407 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:49.080520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:48511 (local address 127.0.0.1:41565)
0808 17:49:49.080557 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:49.080558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:49.080566 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:49:49.080588 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:49.080589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:49.080599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:49.080655 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:49.081156 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:49.081633 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:49.082328 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:49.082622 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:49.082698 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:49.083846 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:49.083848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:49.083848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:49.083855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:49.085216 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:49.085583 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:49.085584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:49.085585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:49.085646 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:49.085862 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:49.085864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:49.085864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:49.085974 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:49.086105 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:49.086215 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 17:49:49.086278 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:49:49.092773 30195 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:49.093289 30195 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:49.105453 30195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c264b969f2514053bad36f10ab91aae4: Generated new TSK 0
I20260808 17:49:49.105655 30195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:49.113417 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30071
W20260808 17:49:49.124248 30049 connection.cc:535] client connection to 127.0.0.1:41565 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41565): Connection reset by peer (error 104)
2026-08-08T17:49:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1541 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:49 dist-test-slave-tk9z krb5kdc[30219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:49 dist-test-slave-tk9z krb5kdc[30219](info): set up 2 sockets
Aug 08 17:49:49 dist-test-slave-tk9z krb5kdc[30219](info): commencing operation
krb5kdc: starting...
W20260808 17:49:50.362169 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:50 dist-test-slave-tk9z krb5kdc[30219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:50.484750 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:50.567207 30239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:50.567471 30239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:50.567512 30239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:50.568665 30239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:50.568732 30239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:50.568758 30239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:50.568787 30239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:50.568817 30239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:50.570363 30239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:50.570664 30239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:50.570884 30239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:50.575690 30246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:50.575682 30253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:50.575685 30248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:50.575876 30239 server_base.cc:1034] running on GCE node
I20260808 17:49:50.576220 30239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:50.576527 30239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:50.577694 30239 hybrid_clock.cc:648] HybridClock initialized: now 1786211390577679 us; error 27 us; skew 500 ppm
Aug 08 17:49:50 dist-test-slave-tk9z krb5kdc[30219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211390, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:50.580219 30239 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:50.580651 30239 webserver.cc:460] Webserver started at http://127.0.0.1:41853/ using document root <none> and password file <none>
I20260808 17:49:50.580878 30239 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:50.580942 30239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:50.581099 30239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:50.582157 30239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "95114095d9554d3eb7236560dc67f643"
format_stamp: "Formatted at 2026-08-08 17:49:50 on dist-test-slave-tk9z"
server_key: "03a562d2167816319526fbe5ff774b2f"
server_key_iv: "f0eab521e9f74da92f7dcfc9ddaa87ee"
server_key_version: "encryptionkey@0"
I20260808 17:49:50.582556 30239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "95114095d9554d3eb7236560dc67f643"
format_stamp: "Formatted at 2026-08-08 17:49:50 on dist-test-slave-tk9z"
server_key: "03a562d2167816319526fbe5ff774b2f"
server_key_iv: "f0eab521e9f74da92f7dcfc9ddaa87ee"
server_key_version: "encryptionkey@0"
I20260808 17:49:50.583873 30239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:50.584570 30267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:50.584739 30239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:49:50.584844 30239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "95114095d9554d3eb7236560dc67f643"
format_stamp: "Formatted at 2026-08-08 17:49:50 on dist-test-slave-tk9z"
server_key: "03a562d2167816319526fbe5ff774b2f"
server_key_iv: "f0eab521e9f74da92f7dcfc9ddaa87ee"
server_key_version: "encryptionkey@0"
I20260808 17:49:50.584916 30239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:50.596599 30239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:50.598543 30239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:50.598695 30239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:50.602784 30239 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44195
I20260808 17:49:50.602830 30342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44195 every 8 connection(s)
I20260808 17:49:50.603204 30239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:50.604069 30343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:50.606421 30343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643: Bootstrap starting.
I20260808 17:49:50.606973 30343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:50.607229 30343 log.cc:824] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:50.607885 30343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643: No bootstrap required, opened a new log
I20260808 17:49:50.609432 30343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95114095d9554d3eb7236560dc67f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260808 17:49:50.609575 30343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:50.609601 30343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95114095d9554d3eb7236560dc67f643, State: Initialized, Role: FOLLOWER
I20260808 17:49:50.609742 30343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95114095d9554d3eb7236560dc67f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260808 17:49:50.609830 30343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:50.609854 30343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:50.609920 30343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:50.609900 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30239
I20260808 17:49:50.610038 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:50.610232 29947 external_mini_cluster.cc:1428] Setting key 298f48f83c523c1bbf0cd1cfd55d6105
I20260808 17:49:50.610635 30343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95114095d9554d3eb7236560dc67f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260808 17:49:50.610750 30343 leader_election.cc:302] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95114095d9554d3eb7236560dc67f643; no voters:
I20260808 17:49:50.610929 30343 leader_election.cc:288] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:50.610956 30350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:50.611145 30350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [term 1 LEADER]: Becoming Leader. State: Replica: 95114095d9554d3eb7236560dc67f643, State: Running, Role: LEADER
I20260808 17:49:50.611274 30343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:50.611275 30350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95114095d9554d3eb7236560dc67f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260808 17:49:50.612582 30353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95114095d9554d3eb7236560dc67f643. Latest consensus state: current_term: 1 leader_uuid: "95114095d9554d3eb7236560dc67f643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95114095d9554d3eb7236560dc67f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } } }
I20260808 17:49:50.612716 30353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:50.612970 30351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95114095d9554d3eb7236560dc67f643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95114095d9554d3eb7236560dc67f643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } } }
I20260808 17:49:50.613042 30351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643 [sys.catalog]: This master's current role is: LEADER
W20260808 17:49:50.613269 30374 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:49:50.613323 30374 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:49:50.613366 30375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:50.613502 30375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:50.615702 30375 catalog_manager.cc:1347] Generated new cluster ID: e3456cbceca24d38abc2b48f6ece210b
I20260808 17:49:50.615756 30375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:50.616596 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:50.610758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:40641 (local address 127.0.0.1:44195)
0808 17:49:50.610946 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:50.610956 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:50.611000 (+ 44us) server_negotiation.cc:407] Connection header received
0808 17:49:50.611131 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:50.611140 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:50.611203 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:50.611635 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:50.611737 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:50.613494 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:50.614485 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:50.615100 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:50.615216 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:50.615887 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:50.615891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:50.615892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:50.616097 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:50.616136 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:50.616153 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:50.616239 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:50.616344 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 17:49:50.616373 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260808 17:49:50.617555 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:50.622761 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:50.619157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:36963 (local address 127.0.0.1:44195)
0808 17:49:50.619188 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:50.619190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:50.619198 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:49:50.619237 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:50.619238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:50.619247 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:50.619305 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:50.619423 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:50.620416 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:50.621379 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:50.621697 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:50.621764 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:50.622505 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:50.622506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:50.622507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:50.622512 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:50.622527 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:50.622532 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:50.622591 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:50.622676 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 17:49:50.622689 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:49:50.623683 30375 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:50.624183 30375 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:50.627439 30375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95114095d9554d3eb7236560dc67f643: Generated new TSK 0
I20260808 17:49:50.627604 30375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:50.631888 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30239
W20260808 17:49:50.645514 30216 connection.cc:535] client connection to 127.0.0.1:44195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44195): Connection reset by peer (error 104)
2026-08-08T17:49:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:50 dist-test-slave-tk9z krb5kdc[30389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:50 dist-test-slave-tk9z krb5kdc[30389](info): set up 2 sockets
Aug 08 17:49:50 dist-test-slave-tk9z krb5kdc[30389](info): commencing operation
krb5kdc: starting...
W20260808 17:49:51.876436 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:51 dist-test-slave-tk9z krb5kdc[30389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:49:52.007620 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:52.091954 30410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:52.092208 30410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:52.092248 30410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:52.093482 30410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:52.093545 30410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:52.093570 30410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:52.093597 30410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:52.093657 30410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:52.095453 30410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39405
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:52.095734 30410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:52.095985 30410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:52.100865 30415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:52.100886 30416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:52.100883 30419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:52.100930 30410 server_base.cc:1034] running on GCE node
I20260808 17:49:52.101306 30410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:52.101643 30410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:52.102802 30410 hybrid_clock.cc:648] HybridClock initialized: now 1786211392102789 us; error 37 us; skew 500 ppm
Aug 08 17:49:52 dist-test-slave-tk9z krb5kdc[30389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:52.105334 30410 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:49:52.105780 30410 webserver.cc:460] Webserver started at http://127.0.0.1:34827/ using document root <none> and password file <none>
I20260808 17:49:52.105994 30410 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:52.106036 30410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:52.106127 30410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:52.107256 30410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "589bf649015442b8a0b23c4a75db27e0"
format_stamp: "Formatted at 2026-08-08 17:49:52 on dist-test-slave-tk9z"
server_key: "21a8061a5874716fa6f83f4bd3ec591d"
server_key_iv: "ef5823261cbb4e643f65076f726f0144"
server_key_version: "encryptionkey@0"
I20260808 17:49:52.107577 30410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "589bf649015442b8a0b23c4a75db27e0"
format_stamp: "Formatted at 2026-08-08 17:49:52 on dist-test-slave-tk9z"
server_key: "21a8061a5874716fa6f83f4bd3ec591d"
server_key_iv: "ef5823261cbb4e643f65076f726f0144"
server_key_version: "encryptionkey@0"
I20260808 17:49:52.108893 30410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:49:52.109622 30431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:52.109819 30410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:52.109905 30410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "589bf649015442b8a0b23c4a75db27e0"
format_stamp: "Formatted at 2026-08-08 17:49:52 on dist-test-slave-tk9z"
server_key: "21a8061a5874716fa6f83f4bd3ec591d"
server_key_iv: "ef5823261cbb4e643f65076f726f0144"
server_key_version: "encryptionkey@0"
I20260808 17:49:52.109963 30410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:52.113344 30410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:52.115232 30410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:52.115341 30410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:52.119169 30410 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39405
I20260808 17:49:52.119220 30514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39405 every 8 connection(s)
I20260808 17:49:52.119525 30410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:52.120510 30515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:52.122792 30515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0: Bootstrap starting.
I20260808 17:49:52.123193 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30410
I20260808 17:49:52.123317 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:52.123438 30515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:52.123550 29947 external_mini_cluster.cc:1428] Setting key 0b822c30725e5b458cd21561f9c67337
I20260808 17:49:52.123792 30515 log.cc:824] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:52.124555 30515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0: No bootstrap required, opened a new log
I20260808 17:49:52.126111 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:52.124103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:48771 (local address 127.0.0.1:39405)
0808 17:49:52.124305 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:52.124315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:52.124347 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:49:52.124810 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:52.124817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:52.124842 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:52.124986 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:52.125671 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:52.125676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:52.125676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:52.125684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:52.125719 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:52.125728 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:52.125812 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:52.125878 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 17:49:52.125933 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260808 17:49:52.126932 30515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589bf649015442b8a0b23c4a75db27e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39405 } }
I20260808 17:49:52.127085 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:52.127115 30515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:52.127167 30515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589bf649015442b8a0b23c4a75db27e0, State: Initialized, Role: FOLLOWER
I20260808 17:49:52.127290 30515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589bf649015442b8a0b23c4a75db27e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39405 } }
I20260808 17:49:52.127381 30515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:52.127425 30515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:52.127476 30515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:52.128023 30515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589bf649015442b8a0b23c4a75db27e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39405 } }
I20260808 17:49:52.128146 30515 leader_election.cc:302] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 589bf649015442b8a0b23c4a75db27e0; no voters:
I20260808 17:49:52.128320 30515 leader_election.cc:288] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:52.128367 30525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:52.128522 30525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [term 1 LEADER]: Becoming Leader. State: Replica: 589bf649015442b8a0b23c4a75db27e0, State: Running, Role: LEADER
I20260808 17:49:52.128629 30515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:52.128649 30525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589bf649015442b8a0b23c4a75db27e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39405 } }
I20260808 17:49:52.129138 30529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "589bf649015442b8a0b23c4a75db27e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589bf649015442b8a0b23c4a75db27e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39405 } } }
I20260808 17:49:52.129268 30529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:52.129122 30531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 589bf649015442b8a0b23c4a75db27e0. Latest consensus state: current_term: 1 leader_uuid: "589bf649015442b8a0b23c4a75db27e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589bf649015442b8a0b23c4a75db27e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39405 } } }
I20260808 17:49:52.129582 30541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:52.129621 30531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:52.129914 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:52.128562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.86:39853 (local address 127.0.0.1:39405)
0808 17:49:52.128597 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:52.128599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:52.128865 (+ 266us) server_negotiation.cc:407] Connection header received
0808 17:49:52.128962 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:52.128963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:52.128968 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:52.129018 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:52.129628 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:52.129629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:52.129629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:52.129634 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:52.129647 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:52.129651 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:52.129682 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:52.129809 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 17:49:52.129830 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:49:52.130189 30541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:52.131912 30541 catalog_manager.cc:1347] Generated new cluster ID: c4bb640e49b14e30bdbcf7010c03dfac
I20260808 17:49:52.131999 30541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:52.146166 30541 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:52.146399 30541 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:52.153486 30541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 589bf649015442b8a0b23c4a75db27e0: Generated new TSK 0
I20260808 17:49:52.153676 30541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:52.159685 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30410
2026-08-08T17:49:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:52 dist-test-slave-tk9z krb5kdc[30562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:52 dist-test-slave-tk9z krb5kdc[30562](info): set up 2 sockets
Aug 08 17:49:52 dist-test-slave-tk9z krb5kdc[30562](info): commencing operation
krb5kdc: starting...
W20260808 17:49:53.451864 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:53 dist-test-slave-tk9z krb5kdc[30562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:49:53.576767 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:36817
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:36817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:53.659564 30576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:53.659776 30576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:53.659798 30576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:53.660907 30576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:53.660940 30576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:53.660954 30576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:53.660967 30576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:53.660979 30576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:53.662424 30576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:36817
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:36817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:53.662624 30576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:53.662796 30576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:53.667553 30584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:53.667564 30583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:53.667762 30576 server_base.cc:1034] running on GCE node
W20260808 17:49:53.667624 30587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:53.668030 30576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:53.668311 30576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:53.669483 30576 hybrid_clock.cc:648] HybridClock initialized: now 1786211393669461 us; error 33 us; skew 500 ppm
Aug 08 17:49:53 dist-test-slave-tk9z krb5kdc[30562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211393, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:53.672355 30576 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:53.672821 30576 webserver.cc:460] Webserver started at http://127.29.62.254:43013/ using document root <none> and password file <none>
I20260808 17:49:53.673072 30576 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:53.673138 30576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:53.673285 30576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:53.674319 30576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "0a0472b5e601462b90297b1bb424b585"
format_stamp: "Formatted at 2026-08-08 17:49:53 on dist-test-slave-tk9z"
server_key: "7439c72cd23c12ea84fb2b700c3b216c"
server_key_iv: "04dc7d65bbb621e6869e3e062f75e610"
server_key_version: "encryptionkey@0"
I20260808 17:49:53.674757 30576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "0a0472b5e601462b90297b1bb424b585"
format_stamp: "Formatted at 2026-08-08 17:49:53 on dist-test-slave-tk9z"
server_key: "7439c72cd23c12ea84fb2b700c3b216c"
server_key_iv: "04dc7d65bbb621e6869e3e062f75e610"
server_key_version: "encryptionkey@0"
I20260808 17:49:53.676070 30576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:53.676790 30599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:53.676965 30576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:53.677073 30576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "0a0472b5e601462b90297b1bb424b585"
format_stamp: "Formatted at 2026-08-08 17:49:53 on dist-test-slave-tk9z"
server_key: "7439c72cd23c12ea84fb2b700c3b216c"
server_key_iv: "04dc7d65bbb621e6869e3e062f75e610"
server_key_version: "encryptionkey@0"
I20260808 17:49:53.677142 30576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:53.692030 30576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:53.694245 30576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:53.694481 30576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:53.698309 30576 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:36817
I20260808 17:49:53.698341 30682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:36817 every 8 connection(s)
I20260808 17:49:53.698732 30576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:53.699671 30683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:53.702018 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30576
I20260808 17:49:53.702152 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:53.702414 29947 external_mini_cluster.cc:1428] Setting key 5e13ed06f81638c0aed1015a26110b46
I20260808 17:49:53.702417 30683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585: Bootstrap starting.
I20260808 17:49:53.703173 30683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:53.703508 30683 log.cc:824] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:53.704363 30683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585: No bootstrap required, opened a new log
I20260808 17:49:53.706795 30683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0472b5e601462b90297b1bb424b585" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36817 } }
I20260808 17:49:53.706983 30683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:53.707034 30683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0472b5e601462b90297b1bb424b585, State: Initialized, Role: FOLLOWER
I20260808 17:49:53.707194 30683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0472b5e601462b90297b1bb424b585" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36817 } }
I20260808 17:49:53.707270 30683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:53.707294 30683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:53.707365 30683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:53.708086 30683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0472b5e601462b90297b1bb424b585" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36817 } }
I20260808 17:49:53.708236 30683 leader_election.cc:302] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a0472b5e601462b90297b1bb424b585; no voters:
I20260808 17:49:53.708422 30683 leader_election.cc:288] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:53.708451 30689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:53.708622 30689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0472b5e601462b90297b1bb424b585, State: Running, Role: LEADER
I20260808 17:49:53.708719 30683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:53.708756 30689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0472b5e601462b90297b1bb424b585" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36817 } }
I20260808 17:49:53.709170 30691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a0472b5e601462b90297b1bb424b585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0472b5e601462b90297b1bb424b585" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36817 } } }
I20260808 17:49:53.709254 30691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:53.709162 30689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a0472b5e601462b90297b1bb424b585. Latest consensus state: current_term: 1 leader_uuid: "0a0472b5e601462b90297b1bb424b585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0472b5e601462b90297b1bb424b585" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 36817 } } }
I20260808 17:49:53.709385 30689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:53.709543 30698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 17:49:53 dist-test-slave-tk9z krb5kdc[30562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:53.710297 30698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:53.711946 30698 catalog_manager.cc:1347] Generated new cluster ID: e06fc518b12c403896551de4cad3106c
I20260808 17:49:53.712010 30698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:53.714428 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:53.702961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37111 (local address 127.29.62.254:36817)
0808 17:49:53.703139 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:53.703148 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:53.703177 (+ 29us) server_negotiation.cc:407] Connection header received
0808 17:49:53.703688 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:53.703711 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:53.703778 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:53.703932 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:53.704896 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:53.705762 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:53.707029 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:53.707618 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:53.707698 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:53.711062 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:53.711066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:53.711066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:53.711080 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:53.712862 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:53.713256 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:53.713259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:53.713259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:53.713347 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:53.713579 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:53.713580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:53.713581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:53.713809 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:53.713966 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:53.714097 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 17:49:53.714199 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260808 17:49:53.719120 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:53.722527 30698 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:53.723313 30698 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:53.728382 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:53.720892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42313 (local address 127.29.62.254:36817)
0808 17:49:53.720934 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:53.720937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:53.722166 (+ 1229us) server_negotiation.cc:407] Connection header received
0808 17:49:53.722226 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:53.722227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:53.722239 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:53.722315 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:53.722980 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:53.723528 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:53.724229 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:53.724456 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:53.724537 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:53.725727 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:53.725728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:53.725728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:53.725735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:53.727193 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:53.727567 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:53.727568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:53.727568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:53.727625 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:53.727846 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:53.727847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:53.727847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:53.727958 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:53.728066 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:53.728206 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 17:49:53.728286 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:49:53.728521 30698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a0472b5e601462b90297b1bb424b585: Generated new TSK 0
I20260808 17:49:53.728718 30698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:53.730685 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30576
W20260808 17:49:53.747077 30560 connection.cc:535] client connection to 127.29.62.254:36817 recv error: Network error: failed to read from TLS socket (remote: 127.29.62.254:36817): Connection reset by peer (error 104)
2026-08-08T17:49:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1568 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:53 dist-test-slave-tk9z krb5kdc[30741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:53 dist-test-slave-tk9z krb5kdc[30741](info): set up 2 sockets
Aug 08 17:49:53 dist-test-slave-tk9z krb5kdc[30741](info): commencing operation
krb5kdc: starting...
W20260808 17:49:54.981902 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:55 dist-test-slave-tk9z krb5kdc[30741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:49:55.108871 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:46603
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:35571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:46603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:55.193166 30762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:55.193387 30762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:55.193413 30762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:55.194689 30762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:55.194731 30762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:55.194746 30762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:55.194759 30762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:55.194800 30762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:55.196336 30762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:35571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:46603
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:46603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:55.196593 30762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:55.196825 30762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:55.202096 30776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:55.202092 30770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:55.202137 30772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:55.202440 30762 server_base.cc:1034] running on GCE node
I20260808 17:49:55.202705 30762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:55.203043 30762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:55.204221 30762 hybrid_clock.cc:648] HybridClock initialized: now 1786211395204195 us; error 70 us; skew 500 ppm
Aug 08 17:49:55 dist-test-slave-tk9z krb5kdc[30741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211395, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:55.209858 30762 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:55.210281 30762 webserver.cc:460] Webserver started at http://127.29.62.254:40703/ using document root <none> and password file <none>
I20260808 17:49:55.210565 30762 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:55.210618 30762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:55.210713 30762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:55.211728 30762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "3a1426060dfb4662a047e88d7383bc44"
format_stamp: "Formatted at 2026-08-08 17:49:55 on dist-test-slave-tk9z"
server_key: "be7ac30db5dc51afb969f0f2bf6f8354"
server_key_iv: "c5343e2af0ac3f284e5d44e27686a8e9"
server_key_version: "encryptionkey@0"
I20260808 17:49:55.212054 30762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "3a1426060dfb4662a047e88d7383bc44"
format_stamp: "Formatted at 2026-08-08 17:49:55 on dist-test-slave-tk9z"
server_key: "be7ac30db5dc51afb969f0f2bf6f8354"
server_key_iv: "c5343e2af0ac3f284e5d44e27686a8e9"
server_key_version: "encryptionkey@0"
I20260808 17:49:55.213318 30762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:55.214039 30785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:55.214231 30762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:49:55.214323 30762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "3a1426060dfb4662a047e88d7383bc44"
format_stamp: "Formatted at 2026-08-08 17:49:55 on dist-test-slave-tk9z"
server_key: "be7ac30db5dc51afb969f0f2bf6f8354"
server_key_iv: "c5343e2af0ac3f284e5d44e27686a8e9"
server_key_version: "encryptionkey@0"
I20260808 17:49:55.214422 30762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:55.223515 30762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:55.225792 30762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:55.225903 30762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:55.229912 30762 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:46603
I20260808 17:49:55.230011 30863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:46603 every 8 connection(s)
I20260808 17:49:55.230352 30762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:55.231333 30865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:55.233628 30865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44: Bootstrap starting.
I20260808 17:49:55.234267 30865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:55.234483 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30762
I20260808 17:49:55.234628 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:55.234596 30865 log.cc:824] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:55.234820 29947 external_mini_cluster.cc:1428] Setting key 9450e9279ff67b859343dad89545a97e
I20260808 17:49:55.235494 30865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44: No bootstrap required, opened a new log
I20260808 17:49:55.237704 30865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1426060dfb4662a047e88d7383bc44" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46603 } }
I20260808 17:49:55.237887 30865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:55.237938 30865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1426060dfb4662a047e88d7383bc44, State: Initialized, Role: FOLLOWER
I20260808 17:49:55.238073 30865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1426060dfb4662a047e88d7383bc44" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46603 } }
I20260808 17:49:55.238174 30865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:55.238235 30865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:55.238289 30865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:55.238888 30865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1426060dfb4662a047e88d7383bc44" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46603 } }
I20260808 17:49:55.239018 30865 leader_election.cc:302] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a1426060dfb4662a047e88d7383bc44; no voters:
I20260808 17:49:55.239185 30865 leader_election.cc:288] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:55.239234 30874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:55.239427 30874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1426060dfb4662a047e88d7383bc44, State: Running, Role: LEADER
I20260808 17:49:55.239472 30865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:55.239806 30874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1426060dfb4662a047e88d7383bc44" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46603 } }
I20260808 17:49:55.240332 30877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a1426060dfb4662a047e88d7383bc44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1426060dfb4662a047e88d7383bc44" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46603 } } }
I20260808 17:49:55.240798 30877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:55.240513 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:55.235372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33327 (local address 127.29.62.254:46603)
0808 17:49:55.235584 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:55.235593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:55.235624 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:49:55.236040 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:55.236046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:55.236091 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:55.236239 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:55.236433 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:55.237315 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:55.238212 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:55.238732 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:55.238857 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:55.239655 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:55.239659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:55.239660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:55.239881 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:55.239930 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:55.239950 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:55.240047 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:55.240192 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 17:49:55.240226 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260808 17:49:55.240703 30876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a1426060dfb4662a047e88d7383bc44. Latest consensus state: current_term: 1 leader_uuid: "3a1426060dfb4662a047e88d7383bc44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1426060dfb4662a047e88d7383bc44" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 46603 } } }
I20260808 17:49:55.241006 30876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:55.241135 30891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:55.241703 30891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:55.241933 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:55.243242 30891 catalog_manager.cc:1347] Generated new cluster ID: ede7a5ebba20426cb1fae76c631043f6
I20260808 17:49:55.243281 30891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:55.247198 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:55.243520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44089 (local address 127.29.62.254:46603)
0808 17:49:55.243556 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:55.243558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:55.243666 (+ 108us) server_negotiation.cc:407] Connection header received
0808 17:49:55.243694 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:55.243695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:55.243705 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:55.243777 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:55.243902 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:55.244636 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:55.245641 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:55.246157 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:55.246235 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:55.246932 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:55.246933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:55.246933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:55.246938 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:55.246955 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:55.246960 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:55.247030 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:55.247105 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 17:49:55.247117 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:49:55.250823 30891 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:55.251470 30891 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:55.257346 30891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a1426060dfb4662a047e88d7383bc44: Generated new TSK 0
I20260808 17:49:55.257552 30891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:55.264427 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30762
W20260808 17:49:55.278932 30736 connection.cc:535] client connection to 127.29.62.254:46603 recv error: Network error: failed to read from TLS socket (remote: 127.29.62.254:46603): Connection reset by peer (error 104)
2026-08-08T17:49:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:55 dist-test-slave-tk9z krb5kdc[30915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:55 dist-test-slave-tk9z krb5kdc[30915](info): set up 2 sockets
Aug 08 17:49:55 dist-test-slave-tk9z krb5kdc[30915](info): commencing operation
krb5kdc: starting...
W20260808 17:49:56.537628 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:56 dist-test-slave-tk9z krb5kdc[30915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:49:56.656260 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34363
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:38329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:34363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:56.738888 30933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:56.739109 30933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:56.739132 30933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:56.740227 30933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:56.740268 30933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:56.740283 30933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:56.740295 30933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:56.740309 30933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:56.741849 30933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:38329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:34363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:56.742112 30933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:56.742362 30933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:56.747213 30946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:56.747207 30948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:56.747243 30944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:56.747416 30933 server_base.cc:1034] running on GCE node
I20260808 17:49:56.747789 30933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:56.748092 30933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:56.749300 30933 hybrid_clock.cc:648] HybridClock initialized: now 1786211396749289 us; error 35 us; skew 500 ppm
Aug 08 17:49:56 dist-test-slave-tk9z krb5kdc[30915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211396, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:56.752097 30933 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:56.752593 30933 webserver.cc:460] Webserver started at http://127.29.62.254:37423/ using document root <none> and password file <none>
I20260808 17:49:56.752841 30933 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:56.752911 30933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:56.753067 30933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:56.754140 30933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "c7871a6769954d6bb70dffef7870a8ac"
format_stamp: "Formatted at 2026-08-08 17:49:56 on dist-test-slave-tk9z"
server_key: "03a76a97147cf90d65b28d057a46a71c"
server_key_iv: "4712578fe59538412de98d47976575eb"
server_key_version: "encryptionkey@0"
I20260808 17:49:56.754583 30933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "c7871a6769954d6bb70dffef7870a8ac"
format_stamp: "Formatted at 2026-08-08 17:49:56 on dist-test-slave-tk9z"
server_key: "03a76a97147cf90d65b28d057a46a71c"
server_key_iv: "4712578fe59538412de98d47976575eb"
server_key_version: "encryptionkey@0"
I20260808 17:49:56.755983 30933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:49:56.756673 30963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:56.756836 30933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:56.756940 30933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "c7871a6769954d6bb70dffef7870a8ac"
format_stamp: "Formatted at 2026-08-08 17:49:56 on dist-test-slave-tk9z"
server_key: "03a76a97147cf90d65b28d057a46a71c"
server_key_iv: "4712578fe59538412de98d47976575eb"
server_key_version: "encryptionkey@0"
I20260808 17:49:56.757026 30933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:56.760716 30933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:56.763051 30933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:56.763209 30933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:56.767417 30933 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:34363
I20260808 17:49:56.767450 31043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:34363 every 8 connection(s)
I20260808 17:49:56.767781 30933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:56.768784 31044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:56.770879 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 30933
I20260808 17:49:56.771027 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:56.771224 29947 external_mini_cluster.cc:1428] Setting key 298d40bd3e56d3274f98a72f506c8d36
I20260808 17:49:56.771445 31044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac: Bootstrap starting.
I20260808 17:49:56.772114 31044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:56.772414 31044 log.cc:824] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:56.773204 31044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac: No bootstrap required, opened a new log
I20260808 17:49:56.773842 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:56.771748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44007 (local address 127.29.62.254:34363)
0808 17:49:56.771964 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:56.771980 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:56.772011 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:49:56.772540 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:56.772549 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:56.772585 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:56.772754 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:56.773385 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:56.773388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:56.773388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:56.773392 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:56.773428 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:56.773438 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:56.773496 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:56.773580 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 17:49:56.773609 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260808 17:49:56.774906 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:56.775704 31044 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7871a6769954d6bb70dffef7870a8ac" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34363 } }
I20260808 17:49:56.775870 31044 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:56.775923 31044 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7871a6769954d6bb70dffef7870a8ac, State: Initialized, Role: FOLLOWER
I20260808 17:49:56.776057 31044 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7871a6769954d6bb70dffef7870a8ac" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34363 } }
I20260808 17:49:56.776180 31044 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:56.776227 31044 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:56.776284 31044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:56.776858 31044 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7871a6769954d6bb70dffef7870a8ac" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34363 } }
I20260808 17:49:56.776984 31044 leader_election.cc:302] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7871a6769954d6bb70dffef7870a8ac; no voters:
I20260808 17:49:56.777150 31044 leader_election.cc:288] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:56.777179 31061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:56.777375 31061 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [term 1 LEADER]: Becoming Leader. State: Replica: c7871a6769954d6bb70dffef7870a8ac, State: Running, Role: LEADER
I20260808 17:49:56.777449 31044 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:56.777529 31061 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7871a6769954d6bb70dffef7870a8ac" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34363 } }
I20260808 17:49:56.777603 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:56.776360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58477 (local address 127.29.62.254:34363)
0808 17:49:56.776400 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:56.776402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:56.776529 (+ 127us) server_negotiation.cc:407] Connection header received
0808 17:49:56.776612 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:56.776613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:56.776618 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:56.776663 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:56.777344 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:56.777346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:56.777346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:56.777351 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:56.777366 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:56.777370 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:56.777401 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:56.777532 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 17:49:56.777546 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:49:56.777957 31061 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7871a6769954d6bb70dffef7870a8ac. Latest consensus state: current_term: 1 leader_uuid: "c7871a6769954d6bb70dffef7870a8ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7871a6769954d6bb70dffef7870a8ac" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34363 } } }
I20260808 17:49:56.778056 31061 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:56.778230 31063 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7871a6769954d6bb70dffef7870a8ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7871a6769954d6bb70dffef7870a8ac" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34363 } } }
I20260808 17:49:56.778668 31075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:56.778688 31063 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:56.779232 31075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:56.783329 31075 catalog_manager.cc:1347] Generated new cluster ID: 9dbe474c2aeb4636b62a271a652dbd51
I20260808 17:49:56.783402 31075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:56.797008 31075 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:56.797191 31075 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:56.803994 31075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7871a6769954d6bb70dffef7870a8ac: Generated new TSK 0
I20260808 17:49:56.804179 31075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:56.808286 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 30933
2026-08-08T17:49:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:56 dist-test-slave-tk9z krb5kdc[31099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:56 dist-test-slave-tk9z krb5kdc[31099](info): set up 2 sockets
Aug 08 17:49:56 dist-test-slave-tk9z krb5kdc[31099](info): commencing operation
krb5kdc: starting...
W20260808 17:49:58.038564 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:58 dist-test-slave-tk9z krb5kdc[31099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:49:58.158774 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:41921
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:45219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:41921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:58.240926 31117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:58.241176 31117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:58.241217 31117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:58.242398 31117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:58.242467 31117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:58.242491 31117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:58.242515 31117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:58.242543 31117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:58.244043 31117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:45219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:41921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:41921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:58.244338 31117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:58.244578 31117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:58.249439 31123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:58.249415 31126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:58.249435 31124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:58.249579 31117 server_base.cc:1034] running on GCE node
I20260808 17:49:58.249924 31117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:58.250219 31117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:58.251368 31117 hybrid_clock.cc:648] HybridClock initialized: now 1786211398251352 us; error 43 us; skew 500 ppm
Aug 08 17:49:58 dist-test-slave-tk9z krb5kdc[31099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211398, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:58.253834 31117 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:58.254304 31117 webserver.cc:460] Webserver started at http://127.29.62.254:36629/ using document root <none> and password file <none>
I20260808 17:49:58.254556 31117 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:58.254602 31117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:58.254694 31117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:58.255729 31117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "fe79961876c44154aa668e43f7fd5756"
format_stamp: "Formatted at 2026-08-08 17:49:58 on dist-test-slave-tk9z"
server_key: "eb195493ede8e324f15b3e9f4f8e9b7a"
server_key_iv: "858b84cd1e85a07201111ed25bfc6269"
server_key_version: "encryptionkey@0"
I20260808 17:49:58.256039 31117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "fe79961876c44154aa668e43f7fd5756"
format_stamp: "Formatted at 2026-08-08 17:49:58 on dist-test-slave-tk9z"
server_key: "eb195493ede8e324f15b3e9f4f8e9b7a"
server_key_iv: "858b84cd1e85a07201111ed25bfc6269"
server_key_version: "encryptionkey@0"
I20260808 17:49:58.257310 31117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:58.257972 31139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:58.258144 31117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:49:58.258252 31117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "fe79961876c44154aa668e43f7fd5756"
format_stamp: "Formatted at 2026-08-08 17:49:58 on dist-test-slave-tk9z"
server_key: "eb195493ede8e324f15b3e9f4f8e9b7a"
server_key_iv: "858b84cd1e85a07201111ed25bfc6269"
server_key_version: "encryptionkey@0"
I20260808 17:49:58.258327 31117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:58.282572 31117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:58.284530 31117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:58.284701 31117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:58.289151 31117 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:41921
I20260808 17:49:58.289211 31223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:41921 every 8 connection(s)
I20260808 17:49:58.289592 31117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:58.290570 31225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:58.292912 31225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756: Bootstrap starting.
I20260808 17:49:58.293560 31225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:58.293640 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31117
I20260808 17:49:58.293779 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:58.293944 31225 log.cc:824] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:58.293972 29947 external_mini_cluster.cc:1428] Setting key c1337eb9c7c2c90edb7114b565a4b150
I20260808 17:49:58.294878 31225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756: No bootstrap required, opened a new log
I20260808 17:49:58.297484 31225 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe79961876c44154aa668e43f7fd5756" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41921 } }
I20260808 17:49:58.297664 31225 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:58.297699 31225 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe79961876c44154aa668e43f7fd5756, State: Initialized, Role: FOLLOWER
I20260808 17:49:58.297814 31225 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe79961876c44154aa668e43f7fd5756" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41921 } }
I20260808 17:49:58.297884 31225 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:58.297912 31225 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:58.297956 31225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:58.298636 31225 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe79961876c44154aa668e43f7fd5756" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41921 } }
I20260808 17:49:58.298728 31225 leader_election.cc:302] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe79961876c44154aa668e43f7fd5756; no voters:
I20260808 17:49:58.298964 31225 leader_election.cc:288] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:58.298985 31232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:58.299111 31232 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [term 1 LEADER]: Becoming Leader. State: Replica: fe79961876c44154aa668e43f7fd5756, State: Running, Role: LEADER
I20260808 17:49:58.299221 31232 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe79961876c44154aa668e43f7fd5756" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41921 } }
I20260808 17:49:58.299250 31225 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:58.299562 31234 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe79961876c44154aa668e43f7fd5756. Latest consensus state: current_term: 1 leader_uuid: "fe79961876c44154aa668e43f7fd5756" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe79961876c44154aa668e43f7fd5756" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41921 } } }
I20260808 17:49:58.299733 31234 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:58.299796 31233 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe79961876c44154aa668e43f7fd5756" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe79961876c44154aa668e43f7fd5756" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 41921 } } }
I20260808 17:49:58.300757 31233 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756 [sys.catalog]: This master's current role is: LEADER
W20260808 17:49:58.301220 31252 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:49:58.301291 31252 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:49:58.301388 31253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:58.301590 31253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:49:58 dist-test-slave-tk9z krb5kdc[31099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:49:58.303158 31253 catalog_manager.cc:1347] Generated new cluster ID: 89c9e639d35548ec9d6d7013305ba9b6
I20260808 17:49:58.303220 31253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:58.306125 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:58.294541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38539 (local address 127.29.62.254:41921)
0808 17:49:58.294739 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:58.294747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:58.294774 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:49:58.295292 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:58.295299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:58.295349 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:58.295462 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:58.296297 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:58.297147 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:58.298243 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:58.298751 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:58.298882 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:58.302237 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:58.302241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:58.302241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:58.302253 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:58.304405 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:58.304813 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:58.304817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:58.304817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:58.304943 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:58.305154 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:58.305155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:58.305156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:58.305464 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:58.305632 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:58.305726 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 17:49:58.305880 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260808 17:49:58.307219 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:58.315622 31253 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:58.316183 31253 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:58.321563 31253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe79961876c44154aa668e43f7fd5756: Generated new TSK 0
I20260808 17:49:58.321759 31253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:58.323158 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:58.315366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51807 (local address 127.29.62.254:41921)
0808 17:49:58.315411 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:58.315413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:58.315723 (+ 310us) server_negotiation.cc:407] Connection header received
0808 17:49:58.315781 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:58.315782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:58.315794 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:58.315825 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:58.316574 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:58.317312 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:58.318351 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:58.318696 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:58.318758 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:58.320214 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:58.320215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:58.320215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:49:58.320222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:58.322035 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:58.322440 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:58.322441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:58.322441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:58.322503 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:49:58.322706 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:49:58.322706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:49:58.322706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:49:58.322804 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:58.322876 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:58.323027 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 17:49:58.323099 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:49:58.325028 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31117
2026-08-08T17:49:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:58 dist-test-slave-tk9z krb5kdc[31267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:58 dist-test-slave-tk9z krb5kdc[31267](info): set up 2 sockets
Aug 08 17:49:58 dist-test-slave-tk9z krb5kdc[31267](info): commencing operation
krb5kdc: starting...
W20260808 17:49:59.558665 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:49:59 dist-test-slave-tk9z krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:49:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:49:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:49:59.679510 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:37439
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:37439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:49:59.764722 31284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:49:59.764921 31284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:49:59.764945 31284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:49:59.766112 31284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:49:59.766158 31284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:49:59.766173 31284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:49:59.766186 31284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:49:59.766239 31284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:49:59.767743 31284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:37439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:37439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:49:59.768003 31284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:49:59.768234 31284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:49:59.773192 31292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:49:59.773226 31293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:59.773260 31284 server_base.cc:1034] running on GCE node
W20260808 17:49:59.773226 31296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:49:59.773667 31284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:49:59.773972 31284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:49:59.775149 31284 hybrid_clock.cc:648] HybridClock initialized: now 1786211399775119 us; error 48 us; skew 500 ppm
Aug 08 17:49:59 dist-test-slave-tk9z krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211399, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:49:59.778014 31284 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:49:59.778630 31284 webserver.cc:460] Webserver started at http://127.29.62.254:33253/ using document root <none> and password file <none>
I20260808 17:49:59.778914 31284 fs_manager.cc:359] Metadata directory not provided
I20260808 17:49:59.778964 31284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:49:59.779102 31284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:49:59.780236 31284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "2b82c934b08a4740ba595b89fddb555a"
format_stamp: "Formatted at 2026-08-08 17:49:59 on dist-test-slave-tk9z"
server_key: "daad960de8f5d5ec2824659c280bbe3e"
server_key_iv: "d21dce0759c371092fced38c35a057f1"
server_key_version: "encryptionkey@0"
I20260808 17:49:59.780632 31284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "2b82c934b08a4740ba595b89fddb555a"
format_stamp: "Formatted at 2026-08-08 17:49:59 on dist-test-slave-tk9z"
server_key: "daad960de8f5d5ec2824659c280bbe3e"
server_key_iv: "d21dce0759c371092fced38c35a057f1"
server_key_version: "encryptionkey@0"
I20260808 17:49:59.782063 31284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:49:59.782986 31309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:49:59.783311 31284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:49:59.783425 31284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "2b82c934b08a4740ba595b89fddb555a"
format_stamp: "Formatted at 2026-08-08 17:49:59 on dist-test-slave-tk9z"
server_key: "daad960de8f5d5ec2824659c280bbe3e"
server_key_iv: "d21dce0759c371092fced38c35a057f1"
server_key_version: "encryptionkey@0"
I20260808 17:49:59.783502 31284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:49:59.803965 31284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:49:59.806586 31284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:49:59.806752 31284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:49:59.810900 31284 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:37439
I20260808 17:49:59.810957 31388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:37439 every 8 connection(s)
I20260808 17:49:59.811303 31284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:49:59.812279 31390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:49:59.814857 31390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a: Bootstrap starting.
I20260808 17:49:59.815428 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31284
I20260808 17:49:59.815448 31390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a: Neither blocks nor log segments found. Creating new log.
I20260808 17:49:59.815541 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:49:59.815718 29947 external_mini_cluster.cc:1428] Setting key f087bc27c2dfffc6020e4fb602219414
I20260808 17:49:59.815816 31390 log.cc:824] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a: Log is configured to *not* fsync() on all Append() calls
I20260808 17:49:59.816641 31390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a: No bootstrap required, opened a new log
I20260808 17:49:59.818310 31390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b82c934b08a4740ba595b89fddb555a" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37439 } }
I20260808 17:49:59.818554 31390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:49:59.818617 31390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b82c934b08a4740ba595b89fddb555a, State: Initialized, Role: FOLLOWER
I20260808 17:49:59.818765 31390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b82c934b08a4740ba595b89fddb555a" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37439 } }
I20260808 17:49:59.818884 31390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:49:59.818930 31390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:49:59.818987 31390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:49:59.819909 31390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b82c934b08a4740ba595b89fddb555a" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37439 } }
I20260808 17:49:59.819993 31390 leader_election.cc:302] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b82c934b08a4740ba595b89fddb555a; no voters:
I20260808 17:49:59.820120 31390 leader_election.cc:288] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:49:59.820178 31398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:49:59.820335 31398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [term 1 LEADER]: Becoming Leader. State: Replica: 2b82c934b08a4740ba595b89fddb555a, State: Running, Role: LEADER
I20260808 17:49:59.820379 31390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:49:59.820438 31398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b82c934b08a4740ba595b89fddb555a" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37439 } }
I20260808 17:49:59.820914 31399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b82c934b08a4740ba595b89fddb555a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b82c934b08a4740ba595b89fddb555a" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37439 } } }
I20260808 17:49:59.820995 31399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:59.821005 31400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b82c934b08a4740ba595b89fddb555a. Latest consensus state: current_term: 1 leader_uuid: "2b82c934b08a4740ba595b89fddb555a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b82c934b08a4740ba595b89fddb555a" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 37439 } } }
I20260808 17:49:59.821117 31400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a [sys.catalog]: This master's current role is: LEADER
I20260808 17:49:59.821306 31409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:49:59.821870 31409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:49:59.823486 31409 catalog_manager.cc:1347] Generated new cluster ID: 348e4e69fe534577b38ed113b3326e5b
I20260808 17:49:59.823586 31409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:49:59.823827 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:59.816261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51757 (local address 127.29.62.254:37439)
0808 17:49:59.816437 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:59.816446 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:59.818637 (+ 2191us) server_negotiation.cc:407] Connection header received
0808 17:49:59.818810 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:59.818817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:59.818869 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:59.819025 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:59.819245 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:59.820215 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:59.821413 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:59.822020 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:59.822101 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:59.822981 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:59.822986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:59.822986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:59.823226 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:59.823266 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:59.823281 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:59.823353 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:59.823553 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 17:49:59.823617 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260808 17:49:59.824735 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:49:59.835034 31409 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:49:59.835832 31409 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:49:59.837617 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:49:59.826170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60359 (local address 127.29.62.254:37439)
0808 17:49:59.826201 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:49:59.826202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:49:59.834640 (+ 8438us) server_negotiation.cc:407] Connection header received
0808 17:49:59.834757 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:49:59.834758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:49:59.834772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:49:59.834847 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:49:59.835069 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:59.835575 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:59.836350 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:49:59.836554 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:49:59.836640 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:49:59.837340 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:49:59.837341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:49:59.837342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:49:59.837346 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:49:59.837363 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:49:59.837367 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:49:59.837410 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 17:49:59.837477 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 17:49:59.837518 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:49:59.844836 31409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b82c934b08a4740ba595b89fddb555a: Generated new TSK 0
I20260808 17:49:59.845069 31409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:49:59.851179 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31284
2026-08-08T17:49:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:49:59 dist-test-slave-tk9z krb5kdc[31434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:49:59 dist-test-slave-tk9z krb5kdc[31434](info): set up 2 sockets
Aug 08 17:49:59 dist-test-slave-tk9z krb5kdc[31434](info): commencing operation
krb5kdc: starting...
W20260808 17:50:01.103480 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:50:01 dist-test-slave-tk9z krb5kdc[31434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:50:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:50:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:50:01.219698 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:39451
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:39451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:50:01.302588 31455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:50:01.302819 31455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:50:01.302855 31455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:50:01.304000 31455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:50:01.304065 31455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:50:01.304090 31455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:50:01.304122 31455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:50:01.304149 31455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:50:01.305640 31455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:39451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:39451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:50:01.305912 31455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:50:01.306142 31455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:50:01.311020 31464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:01.311044 31455 server_base.cc:1034] running on GCE node
W20260808 17:50:01.311060 31466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:01.311193 31469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:01.311407 31455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:50:01.311707 31455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:50:01.312882 31455 hybrid_clock.cc:648] HybridClock initialized: now 1786211401312860 us; error 38 us; skew 500 ppm
Aug 08 17:50:01 dist-test-slave-tk9z krb5kdc[31434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211401, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:50:01.318655 31455 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:50:01.319136 31455 webserver.cc:460] Webserver started at http://127.29.62.254:41871/ using document root <none> and password file <none>
I20260808 17:50:01.319360 31455 fs_manager.cc:359] Metadata directory not provided
I20260808 17:50:01.319401 31455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:50:01.319494 31455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:50:01.320520 31455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "3532b6fc5f234c6ebc91867cc5378055"
format_stamp: "Formatted at 2026-08-08 17:50:01 on dist-test-slave-tk9z"
server_key: "728c556883c7950d8d5ea27e565420d9"
server_key_iv: "28567e73c2f28721ab86490507152c81"
server_key_version: "encryptionkey@0"
I20260808 17:50:01.320832 31455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "3532b6fc5f234c6ebc91867cc5378055"
format_stamp: "Formatted at 2026-08-08 17:50:01 on dist-test-slave-tk9z"
server_key: "728c556883c7950d8d5ea27e565420d9"
server_key_iv: "28567e73c2f28721ab86490507152c81"
server_key_version: "encryptionkey@0"
I20260808 17:50:01.322103 31455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:50:01.322866 31480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:50:01.323074 31455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:50:01.323172 31455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "3532b6fc5f234c6ebc91867cc5378055"
format_stamp: "Formatted at 2026-08-08 17:50:01 on dist-test-slave-tk9z"
server_key: "728c556883c7950d8d5ea27e565420d9"
server_key_iv: "28567e73c2f28721ab86490507152c81"
server_key_version: "encryptionkey@0"
I20260808 17:50:01.323248 31455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:50:01.326507 31455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:50:01.328248 31455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:50:01.328400 31455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:50:01.332470 31455 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:39451
I20260808 17:50:01.332501 31563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:39451 every 8 connection(s)
I20260808 17:50:01.332819 31455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:50:01.333801 31565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:50:01.334662 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31455
I20260808 17:50:01.334795 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:50:01.334987 29947 external_mini_cluster.cc:1428] Setting key 58a67f42a9edbf27a77488547c7e0af3
I20260808 17:50:01.337132 31565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055: Bootstrap starting.
I20260808 17:50:01.337512 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:01.335490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54447 (local address 127.29.62.254:39451)
0808 17:50:01.335681 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:01.335690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:01.335723 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:50:01.336249 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:01.336258 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:01.336296 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:01.336446 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:01.337080 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:01.337083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:01.337083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:50:01.337087 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:01.337129 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:50:01.337145 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:01.337223 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:01.337283 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 17:50:01.337323 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260808 17:50:01.337863 31565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055: Neither blocks nor log segments found. Creating new log.
I20260808 17:50:01.338181 31565 log.cc:824] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055: Log is configured to *not* fsync() on all Append() calls
I20260808 17:50:01.338591 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:50:01.338963 31565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055: No bootstrap required, opened a new log
I20260808 17:50:01.341050 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:01.340042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42257 (local address 127.29.62.254:39451)
0808 17:50:01.340099 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:01.340101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:01.340183 (+ 82us) server_negotiation.cc:407] Connection header received
0808 17:50:01.340228 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:01.340229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:01.340233 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:01.340303 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:01.340848 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:01.340849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:01.340850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:50:01.340855 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:01.340871 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:50:01.340877 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:01.340947 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:01.340981 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 17:50:01.340992 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:50:01.341266 31565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3532b6fc5f234c6ebc91867cc5378055" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39451 } }
I20260808 17:50:01.341429 31565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:50:01.341478 31565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3532b6fc5f234c6ebc91867cc5378055, State: Initialized, Role: FOLLOWER
I20260808 17:50:01.341595 31565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3532b6fc5f234c6ebc91867cc5378055" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39451 } }
I20260808 17:50:01.341684 31565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:50:01.341727 31565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:50:01.341779 31565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:50:01.342300 31565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3532b6fc5f234c6ebc91867cc5378055" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39451 } }
I20260808 17:50:01.342437 31565 leader_election.cc:302] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3532b6fc5f234c6ebc91867cc5378055; no voters:
I20260808 17:50:01.342610 31565 leader_election.cc:288] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:50:01.342672 31582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:50:01.342821 31582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [term 1 LEADER]: Becoming Leader. State: Replica: 3532b6fc5f234c6ebc91867cc5378055, State: Running, Role: LEADER
I20260808 17:50:01.342850 31565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:50:01.342944 31582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3532b6fc5f234c6ebc91867cc5378055" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39451 } }
I20260808 17:50:01.343348 31583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3532b6fc5f234c6ebc91867cc5378055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3532b6fc5f234c6ebc91867cc5378055" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39451 } } }
I20260808 17:50:01.343555 31587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3532b6fc5f234c6ebc91867cc5378055. Latest consensus state: current_term: 1 leader_uuid: "3532b6fc5f234c6ebc91867cc5378055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3532b6fc5f234c6ebc91867cc5378055" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39451 } } }
I20260808 17:50:01.343638 31583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [sys.catalog]: This master's current role is: LEADER
I20260808 17:50:01.343890 31587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055 [sys.catalog]: This master's current role is: LEADER
I20260808 17:50:01.344129 31605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:50:01.344691 31605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:50:01.346226 31605 catalog_manager.cc:1347] Generated new cluster ID: 82c007560a5e490a8fdf9677318bbad0
I20260808 17:50:01.346283 31605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:50:01.354341 31605 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:50:01.354537 31605 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:50:01.357726 31605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3532b6fc5f234c6ebc91867cc5378055: Generated new TSK 0
I20260808 17:50:01.357877 31605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:50:01.361017 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31455
2026-08-08T17:50:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1514 ms)
[----------] 18 tests from AuthTokenIssuingTest (27640 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:50:01 dist-test-slave-tk9z krb5kdc[31619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:50:01 dist-test-slave-tk9z krb5kdc[31619](info): set up 2 sockets
Aug 08 17:50:01 dist-test-slave-tk9z krb5kdc[31619](info): commencing operation
krb5kdc: starting...
W20260808 17:50:02.619386 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:50:02 dist-test-slave-tk9z krb5kdc[31619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:50:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:50:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:50:02.743055 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:39349
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:39349,127.29.62.253:42873,127.29.62.252:37627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:50:02.826771 31636 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:50:02.827039 31636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:50:02.827071 31636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:50:02.827129 31636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:50:02.828418 31636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:50:02.828481 31636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:50:02.828504 31636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:50:02.828527 31636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:50:02.828555 31636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:50:02.830096 31636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:39349,127.29.62.253:42873,127.29.62.252:37627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:39349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:50:02.830415 31636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:50:02.830657 31636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:50:02.835549 31645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:02.835686 31649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:02.835709 31647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:02.835767 31636 server_base.cc:1034] running on GCE node
I20260808 17:50:02.836135 31636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:50:02.836462 31636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:50:02.837631 31636 hybrid_clock.cc:648] HybridClock initialized: now 1786211402837615 us; error 35 us; skew 500 ppm
Aug 08 17:50:02 dist-test-slave-tk9z krb5kdc[31619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211402, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:50:02.840215 31636 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:50:02.840662 31636 webserver.cc:460] Webserver started at http://127.29.62.254:33937/ using document root <none> and password file <none>
I20260808 17:50:02.840874 31636 fs_manager.cc:359] Metadata directory not provided
I20260808 17:50:02.840924 31636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:50:02.841012 31636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:50:02.842036 31636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "d6391e6017354812a7c36c0732013ad0"
format_stamp: "Formatted at 2026-08-08 17:50:02 on dist-test-slave-tk9z"
server_key: "d821e518d7c2468e1f2df17423d51aa3"
server_key_iv: "1c90e38fa9e32d20ac7cac25a8d6d759"
server_key_version: "encryptionkey@0"
I20260808 17:50:02.842344 31636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "d6391e6017354812a7c36c0732013ad0"
format_stamp: "Formatted at 2026-08-08 17:50:02 on dist-test-slave-tk9z"
server_key: "d821e518d7c2468e1f2df17423d51aa3"
server_key_iv: "1c90e38fa9e32d20ac7cac25a8d6d759"
server_key_version: "encryptionkey@0"
I20260808 17:50:02.843670 31636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:50:02.844348 31656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:50:02.844509 31636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:50:02.844599 31636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "d6391e6017354812a7c36c0732013ad0"
format_stamp: "Formatted at 2026-08-08 17:50:02 on dist-test-slave-tk9z"
server_key: "d821e518d7c2468e1f2df17423d51aa3"
server_key_iv: "1c90e38fa9e32d20ac7cac25a8d6d759"
server_key_version: "encryptionkey@0"
I20260808 17:50:02.844671 31636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:50:02.874974 31636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:50:02.877418 31636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:50:02.877537 31636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:50:02.881755 31636 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:39349
I20260808 17:50:02.881812 31736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:39349 every 8 connection(s)
I20260808 17:50:02.882195 31636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:50:02.883394 31737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:50:02.885076 31737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:02.888922 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31636
I20260808 17:50:02.889057 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:50:02.889278 29947 external_mini_cluster.cc:1428] Setting key f20bcf32fde86ca43507db5e09ff3089
Aug 08 17:50:02 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211402, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:50:02.895264 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:02.885918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.29.62.254:39349)
0808 17:50:02.886062 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:02.886072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:02.886202 (+ 130us) server_negotiation.cc:407] Connection header received
0808 17:50:02.886839 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:02.886840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:02.886857 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:02.886953 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:02.887535 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:02.888340 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:02.889313 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:02.889663 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:02.889782 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:02.891912 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:02.891923 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:02.891924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:02.891930 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:02.893526 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:02.893948 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:02.893951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:02.893951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:02.894036 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:02.894298 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:02.894299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:02.894299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:02.894577 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:02.894669 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:02.894867 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 17:50:02.895012 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260808 17:50:02.895267 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:02.885762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:39349 (local address 127.0.0.1:47248)
0808 17:50:02.886050 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:02.886097 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:02.886672 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:02.886961 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:02.886963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:02.887264 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:02.887412 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:02.887417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:02.888444 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:02.888445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:02.889222 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:02.889224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:02.889765 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:02.889766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:02.889830 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:02.890401 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:02.890411 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:02.891807 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:02.893630 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:02.893637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:02.893639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:02.893848 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:02.894106 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:02.894107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:02.894107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:02.894202 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:02.894649 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:02.894655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:02.894721 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 17:50:02.894833 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 17:50:02.895079 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":98,"thread_start_us":126,"threads_started":1}
I20260808 17:50:02.896160 31737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:02.896541 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:02.896417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60418)
0808 17:50:02.896466 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:02.896520 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:50:02.896704 31660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
I20260808 17:50:02.897151 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:02.897058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60432)
0808 17:50:02.897103 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:02.897128 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 17:50:02.897259 31737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:42873: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.62.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab.
Entry for principal kudu/127.29.62.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab.
I20260808 17:50:02.917366 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.253:42873
--webserver_interface=127.29.62.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab
--principal=kudu/127.29.62.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:39349,127.29.62.253:42873,127.29.62.252:37627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:50:02.946579 31737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } attempt: 1
I20260808 17:50:02.947135 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:02.946980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60448)
0808 17:50:02.947057 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:02.947101 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 17:50:02.947590 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:02.947489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60456)
0808 17:50:02.947541 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:02.947570 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 17:50:02.947690 31737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:42873: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
W20260808 17:50:03.004107 31749 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:50:03.004328 31749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:50:03.004347 31749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:50:03.004369 31749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:50:03.005541 31749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:50:03.005587 31749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:50:03.005602 31749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:50:03.005615 31749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:50:03.005627 31749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:50:03.007202 31749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:39349,127.29.62.253:42873,127.29.62.252:37627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab
--principal=kudu/127.29.62.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.253:42873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.62.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:50:03.007470 31749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:50:03.007735 31749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:50:03.012730 31760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:03.012753 31761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:03.012775 31749 server_base.cc:1034] running on GCE node
W20260808 17:50:03.012840 31765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:03.013156 31749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:50:03.013494 31749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:50:03.014643 31749 hybrid_clock.cc:648] HybridClock initialized: now 1786211403014627 us; error 29 us; skew 500 ppm
I20260808 17:50:03.015826 31737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } attempt: 2
I20260808 17:50:03.016234 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.016103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60462)
0808 17:50:03.016165 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.016204 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:50:03.016567 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.016457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60464)
0808 17:50:03.016516 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.016543 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 17:50:03.016655 31737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:42873: Network error: Client connection negotiation failed: client connection to 127.29.62.253:42873: connect: Connection refused (error 111)
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211403, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:50:03.017704 31749 init.cc:377] Logged in from keytab as kudu/127.29.62.253@KRBTEST.COM (short username kudu)
I20260808 17:50:03.018124 31749 webserver.cc:460] Webserver started at http://127.29.62.253:41331/ using document root <none> and password file <none>
I20260808 17:50:03.018329 31749 fs_manager.cc:359] Metadata directory not provided
I20260808 17:50:03.018414 31749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:50:03.018577 31749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:50:03.019654 31749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/data/instance:
uuid: "c2dadfa4e5a449cc97000bb0b824ed74"
format_stamp: "Formatted at 2026-08-08 17:50:03 on dist-test-slave-tk9z"
server_key: "ba044721540fd7a55d013ee536e0f88f"
server_key_iv: "7eac854f45f252fa11fa5f2d03cb34cc"
server_key_version: "encryptionkey@0"
I20260808 17:50:03.020030 31749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/wal/instance:
uuid: "c2dadfa4e5a449cc97000bb0b824ed74"
format_stamp: "Formatted at 2026-08-08 17:50:03 on dist-test-slave-tk9z"
server_key: "ba044721540fd7a55d013ee536e0f88f"
server_key_iv: "7eac854f45f252fa11fa5f2d03cb34cc"
server_key_version: "encryptionkey@0"
I20260808 17:50:03.021396 31749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:50:03.022152 31777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:50:03.022320 31749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:50:03.022442 31749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/wal
uuid: "c2dadfa4e5a449cc97000bb0b824ed74"
format_stamp: "Formatted at 2026-08-08 17:50:03 on dist-test-slave-tk9z"
server_key: "ba044721540fd7a55d013ee536e0f88f"
server_key_iv: "7eac854f45f252fa11fa5f2d03cb34cc"
server_key_version: "encryptionkey@0"
I20260808 17:50:03.022531 31749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:50:03.029748 31749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:50:03.031525 31749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:50:03.031630 31749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:50:03.035634 31749 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.253:42873
I20260808 17:50:03.035699 31850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.253:42873 every 8 connection(s)
I20260808 17:50:03.036031 31749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
I20260808 17:50:03.037035 31854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:50:03.038550 31854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:03.042882 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31749
I20260808 17:50:03.043021 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-1/wal/instance
I20260808 17:50:03.043221 29947 external_mini_cluster.cc:1428] Setting key 902e6d0b7e25fd8f772b14cf1ccad2a5
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211403, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:50:03.047794 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.039203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.29.62.254:39349)
0808 17:50:03.039263 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.039265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.039475 (+ 210us) server_negotiation.cc:407] Connection header received
0808 17:50:03.039938 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.039939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.039952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.040010 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.040607 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.041103 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.042043 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.042239 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.042290 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.044490 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.044491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.044492 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.044500 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.045912 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.046489 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.046491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.046491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.046559 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.046931 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.046932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.046933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.047086 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.047168 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.047557 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.047651 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:50:03.047787 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.039173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:39349 (local address 127.0.0.1:47256)
0808 17:50:03.039352 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.039392 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:03.039838 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:03.040082 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:03.040086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:03.040318 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:03.040529 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.040533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.041182 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.041183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.041969 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.041972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.042320 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.042321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.042492 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.042964 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:03.042972 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:03.044401 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:03.045997 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.046005 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.046008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.046359 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.046666 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.046668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.046668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.046840 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.047180 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:03.047187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:03.047411 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 17:50:03.047510 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 17:50:03.047631 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260808 17:50:03.048441 31854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211403, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.62.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.252@KRBTEST.COM" created.
I20260808 17:50:03.056332 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.048720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60476)
0808 17:50:03.048945 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.048956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:03.049009 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:03.049178 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:03.049179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:03.049348 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:03.049421 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.049423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.050235 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.050236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.051156 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.051159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.051637 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.051639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.051726 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.052113 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:03.052118 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:03.053400 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:03.055257 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.055258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.055258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.055422 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.055642 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.055643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.055643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.055698 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.055979 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:03.055980 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:03.056038 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 17:50:03.056104 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:50:03.056177 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260808 17:50:03.056465 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.048799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.29.62.253:42873)
0808 17:50:03.048945 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.048949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.049002 (+ 53us) server_negotiation.cc:407] Connection header received
0808 17:50:03.049067 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.049071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.049086 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.049164 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.049493 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.050138 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.051246 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.051553 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.051618 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.053523 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.053531 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.053531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.053538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.055180 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.055504 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.055506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.055507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.055562 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.055755 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.055756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.055756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.055891 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.056004 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.056157 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.056250 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260808 17:50:03.057085 31854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:03.057428 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.057312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33124)
0808 17:50:03.057359 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.057406 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 17:50:03.057564 31782 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
I20260808 17:50:03.057996 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.057872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33140)
0808 17:50:03.057927 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.057964 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 17:50:03.058175 31854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37627: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab.
Entry for principal kudu/127.29.62.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab.
I20260808 17:50:03.073596 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.252:37627
--webserver_interface=127.29.62.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab
--principal=kudu/127.29.62.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:42027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:39349,127.29.62.253:42873,127.29.62.252:37627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:50:03.107409 31854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } attempt: 1
I20260808 17:50:03.107798 31737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } attempt: 3
I20260808 17:50:03.108000 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.107817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33146)
0808 17:50:03.107902 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.107957 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 17:50:03.108413 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.108282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33148)
0808 17:50:03.108339 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.108374 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 17:50:03.108542 31854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37627: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211402, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
I20260808 17:50:03.116359 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.108147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60484)
0808 17:50:03.108236 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.108247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:03.108313 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:03.108597 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:03.108599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:03.108770 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:03.108839 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.108841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.109849 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.109851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.110576 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.110579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.111123 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.111124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.111245 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.111862 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:03.111868 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:03.113587 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:03.115429 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.115430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.115430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.115601 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.115805 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.115806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.115806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.115860 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.116118 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:03.116119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:03.116159 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 17:50:03.116210 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:50:03.116290 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 17:50:03.116434 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.108319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60484 (local address 127.29.62.253:42873)
0808 17:50:03.108384 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.108386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.108396 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:50:03.108426 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.108427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.108439 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.108571 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.108924 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.109724 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.110666 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.111041 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.111122 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.113694 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.113695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.113696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.113705 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.115332 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.115673 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.115674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.115675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.115739 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.115919 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.115920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.115920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.116057 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.116123 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.116254 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.116339 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:50:03.116832 31737 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:03.117146 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.117060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33156)
0808 17:50:03.117094 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.117125 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 17:50:03.117452 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.117384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33164)
0808 17:50:03.117419 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.117439 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 17:50:03.117558 31737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37627: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
I20260808 17:50:03.148702 31737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } attempt: 1
I20260808 17:50:03.149207 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.149061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33170)
0808 17:50:03.149136 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.149175 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 17:50:03.149624 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.149485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33186)
0808 17:50:03.149549 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.149593 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 17:50:03.149763 31737 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37627: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
W20260808 17:50:03.161197 31865 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:50:03.161406 31865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:50:03.161424 31865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:50:03.161475 31865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:50:03.162662 31865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:50:03.162701 31865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:50:03.162716 31865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:50:03.162760 31865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:50:03.162783 31865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:50:03.164314 31865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:42027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:39349,127.29.62.253:42873,127.29.62.252:37627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab
--principal=kudu/127.29.62.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.252:37627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.62.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:50:03.164575 31865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:50:03.164815 31865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:50:03.169572 31872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:03.169651 31865 server_base.cc:1034] running on GCE node
W20260808 17:50:03.169597 31878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:03.169590 31875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:03.170051 31865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:50:03.170418 31865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:50:03.171578 31865 hybrid_clock.cc:648] HybridClock initialized: now 1786211403171558 us; error 37 us; skew 500 ppm
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211403, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:50:03.174139 31865 init.cc:377] Logged in from keytab as kudu/127.29.62.252@KRBTEST.COM (short username kudu)
I20260808 17:50:03.174669 31865 webserver.cc:460] Webserver started at http://127.29.62.252:45675/ using document root <none> and password file <none>
I20260808 17:50:03.174898 31865 fs_manager.cc:359] Metadata directory not provided
I20260808 17:50:03.174944 31865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:50:03.175107 31865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:50:03.176172 31865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/data/instance:
uuid: "53d89d265e2642eca295a79117a5801c"
format_stamp: "Formatted at 2026-08-08 17:50:03 on dist-test-slave-tk9z"
server_key: "2ff87a95e505e77f69a7f97598a3f490"
server_key_iv: "4fdfa734824b0d3e994e58bb7e56f59d"
server_key_version: "encryptionkey@0"
I20260808 17:50:03.176556 31865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/wal/instance:
uuid: "53d89d265e2642eca295a79117a5801c"
format_stamp: "Formatted at 2026-08-08 17:50:03 on dist-test-slave-tk9z"
server_key: "2ff87a95e505e77f69a7f97598a3f490"
server_key_iv: "4fdfa734824b0d3e994e58bb7e56f59d"
server_key_version: "encryptionkey@0"
I20260808 17:50:03.176710 31854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } attempt: 2
I20260808 17:50:03.177168 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.177029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33198)
0808 17:50:03.177098 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.177138 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 17:50:03.177589 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.177491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33212)
0808 17:50:03.177542 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.177571 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 17:50:03.177711 31854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37627: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37627: connect: Connection refused (error 111)
I20260808 17:50:03.178263 31865 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 17:50:03.179080 31887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:50:03.179327 31865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:50:03.179427 31865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/wal
uuid: "53d89d265e2642eca295a79117a5801c"
format_stamp: "Formatted at 2026-08-08 17:50:03 on dist-test-slave-tk9z"
server_key: "2ff87a95e505e77f69a7f97598a3f490"
server_key_iv: "4fdfa734824b0d3e994e58bb7e56f59d"
server_key_version: "encryptionkey@0"
I20260808 17:50:03.179486 31865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:50:03.192714 31865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:50:03.194674 31865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:50:03.194785 31865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:50:03.198971 31865 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.252:37627
I20260808 17:50:03.199038 31967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.252:37627 every 8 connection(s)
I20260808 17:50:03.199417 31865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
I20260808 17:50:03.199563 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 31865
I20260808 17:50:03.199678 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786211125631729-29947-0/minicluster-data/master-2/wal/instance
I20260808 17:50:03.199918 29947 external_mini_cluster.cc:1428] Setting key 05d250bfcf2fcd55438dd35fb289deba
I20260808 17:50:03.200810 31968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:50:03.202872 31968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211403, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:50:03.210898 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.200514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.29.62.252:37627)
0808 17:50:03.200719 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.200734 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.200759 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:50:03.201251 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.201258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.201305 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.201421 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.202273 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.203232 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.204271 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.204673 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.204800 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.207196 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.207203 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.207203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.207211 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.208856 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.209548 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.209551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.209552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.209659 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.209839 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.209840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.209840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.210164 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.210316 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.210501 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.210652 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260808 17:50:03.211400 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.203599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47260 (local address 127.29.62.254:39349)
0808 17:50:03.203672 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.203674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.203807 (+ 133us) server_negotiation.cc:407] Connection header received
0808 17:50:03.203866 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.203867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.203875 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.203921 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.204465 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.204931 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.205731 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.205944 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.206024 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.208479 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.208480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.208480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.208488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.210076 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.210562 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.210564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.210564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.210629 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.210833 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.210834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.210834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.210985 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.211056 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.211233 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.211313 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:50:03.211423 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.203531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:39349 (local address 127.0.0.1:47260)
0808 17:50:03.203702 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.203744 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:03.203815 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:03.203939 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:03.203941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:03.204287 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:03.204377 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.204380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.205027 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.205028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.205602 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.205604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.206056 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.206057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.206136 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.206678 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:03.206687 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:03.208366 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:03.210154 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.210160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.210161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.210488 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.210714 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.210714 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.210715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.210769 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.211050 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:03.211054 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:03.211102 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 17:50:03.211204 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 17:50:03.211313 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260808 17:50:03.212307 31968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211403, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:50:03.218825 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.212630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.29.62.253:42873)
0808 17:50:03.212671 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.212673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.212680 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:50:03.212705 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.212706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.212721 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.212790 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.213063 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.213520 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.214158 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.214429 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.214489 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.216232 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.216233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.216234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.216241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.217773 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.218112 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.218113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.218114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.218176 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.218323 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.218323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.218324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.218487 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.218564 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.218673 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.218748 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:50:03.218848 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.212554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:42873 (local address 127.0.0.1:60492)
0808 17:50:03.212603 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.212613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:03.212652 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:03.212796 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:03.212797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:03.212937 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:03.212979 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.212980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.213600 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.213601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.214073 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.214075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.214488 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.214489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.214561 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.214925 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:03.214929 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:03.216119 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:03.217855 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.217856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.217857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.218036 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.218242 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.218242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.218242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.218283 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.218554 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:03.218555 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:03.218595 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 17:50:03.218648 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 17:50:03.218744 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:50:03.219415 31968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:03.219967 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.211839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.29.62.254:39349)
0808 17:50:03.211905 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.211908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.211915 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:50:03.211939 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.211940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.211951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.212020 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.212738 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.213179 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.213831 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.214036 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.214110 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.216861 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.216862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.216862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.216869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.218661 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.219046 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.219047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.219048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.219115 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.219338 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.219340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.219340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.219471 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.219670 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.219793 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.219871 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211403, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
I20260808 17:50:03.224941 31737 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } attempt: 2
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
I20260808 17:50:03.227442 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.219687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33228)
0808 17:50:03.219728 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.219737 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:03.219773 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:03.219972 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:03.219972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:03.220134 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:03.220167 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.220168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.221006 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.221007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.221636 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.221639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.222120 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.222120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.222177 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.222587 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:03.222592 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:03.224016 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:03.226027 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.226029 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.226029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.226616 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.226819 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.226820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.226820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.226875 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.227136 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:03.227137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:03.227173 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 17:50:03.227235 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:50:03.227326 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 17:50:03.227802 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.219727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.29.62.252:37627)
0808 17:50:03.219762 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.219764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.219772 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:50:03.219838 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.219839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.219848 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.219895 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.220244 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.220933 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.221708 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.222046 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.222096 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.224097 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.224098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.224099 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.224104 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.225964 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.226692 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.226693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.226694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.226745 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.226956 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.226957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.226957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.227063 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.227129 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.227614 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.227694 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:50:03.229544 31968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c: Bootstrap starting.
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211402, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
I20260808 17:50:03.230449 31968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c: Neither blocks nor log segments found. Creating new log.
I20260808 17:50:03.230654 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.220681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.29.62.253:42873)
0808 17:50:03.220719 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.220721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.220745 (+ 24us) server_negotiation.cc:407] Connection header received
0808 17:50:03.220764 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.220764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.220772 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.220830 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.221360 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.221844 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.222794 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.222972 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.223024 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.225607 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.225608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.225609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.225616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.227281 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.227690 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.227691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.227691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.227749 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.227943 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.227943 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.227944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.228053 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.228200 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.230491 (+ 2291us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.230581 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:50:03.230792 31968 log.cc:824] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:50:03.231961 31968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c: No bootstrap required, opened a new log
I20260808 17:50:03.233641 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.225197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33232)
0808 17:50:03.225235 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.225246 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:03.225287 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:03.225668 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:03.225670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:03.225844 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:03.225891 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.225892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.227366 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.227367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.228157 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.228160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.228830 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.228831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.228877 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.229269 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:03.229274 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:03.230675 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:03.232567 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.232568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.232568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.232734 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.232979 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.232980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.232981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.233042 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.233319 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:03.233320 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:03.233367 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 17:50:03.233438 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 17:50:03.233542 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 17:50:03.233665 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.225299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33232 (local address 127.29.62.252:37627)
0808 17:50:03.225525 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.225527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.225536 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:50:03.225557 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.225557 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.225567 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.225961 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.225993 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.227256 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.228437 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.228747 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.228830 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.230777 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.230779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.230779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.230798 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.232486 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.232830 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.232831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.232832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.232910 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.233115 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.233116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.233116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.233249 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.233326 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.233485 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.233559 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":120,"threads_started":1}
I20260808 17:50:03.233750 31968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } }
I20260808 17:50:03.233942 31968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d89d265e2642eca295a79117a5801c, State: Initialized, Role: FOLLOWER
I20260808 17:50:03.234051 31968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } }
I20260808 17:50:03.234344 31968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:50:03.234336 31976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.234486 31976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:03.235219 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:50:03.235307 31737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0: Bootstrap starting.
I20260808 17:50:03.236018 31737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0: Neither blocks nor log segments found. Creating new log.
I20260808 17:50:03.236328 31737 log.cc:824] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0: Log is configured to *not* fsync() on all Append() calls
W20260808 17:50:03.236647 31990 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:50:03.236696 31990 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:50:03.237358 31737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0: No bootstrap required, opened a new log
I20260808 17:50:03.236312 31697 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d6391e6017354812a7c36c0732013ad0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47274
I20260808 17:50:03.238713 31737 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } }
I20260808 17:50:03.238847 31737 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6391e6017354812a7c36c0732013ad0, State: Initialized, Role: FOLLOWER
I20260808 17:50:03.238931 31737 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } }
I20260808 17:50:03.239249 31737 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:50:03.239288 31697 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 17:50:03.237123 31865 master_runner.cc:186] Error getting master registration for 127.29.62.254:39349: OK, instance_id { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" instance_seqno: 1786211402880605 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:50:03.239332 31697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:50:03.239367 31865 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:50:03.240351 31697 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } }
I20260808 17:50:03.240617 31697 leader_election.cc:288] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [CANDIDATE]: Term 1 election: Requested vote from peers c2dadfa4e5a449cc97000bb0b824ed74 (127.29.62.253:42873), 53d89d265e2642eca295a79117a5801c (127.29.62.252:37627)
I20260808 17:50:03.240911 31926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6391e6017354812a7c36c0732013ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "53d89d265e2642eca295a79117a5801c"
I20260808 17:50:03.241030 31926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [term 0 FOLLOWER]: Advancing to term 1
W20260808 17:50:03.241545 32006 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:50:03.241612 32006 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:50:03.241909 31926 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6391e6017354812a7c36c0732013ad0 in term 1.
I20260808 17:50:03.239228 31993 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.242125 31660 leader_election.cc:302] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53d89d265e2642eca295a79117a5801c, d6391e6017354812a7c36c0732013ad0; no voters:
I20260808 17:50:03.242152 31993 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:03.242213 31993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:50:03.242352 31993 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [term 1 LEADER]: Becoming Leader. State: Replica: d6391e6017354812a7c36c0732013ad0, State: Running, Role: LEADER
I20260808 17:50:03.242468 31993 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } }
I20260808 17:50:03.242923 32021 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6391e6017354812a7c36c0732013ad0. Latest consensus state: current_term: 1 leader_uuid: "d6391e6017354812a7c36c0732013ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.242993 32021 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:50:03.243423 32031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:50:03.243590 32031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:50:03.242714 31636 master_runner.cc:186] Error getting master registration for 127.29.62.253:42873: OK, instance_id { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" instance_seqno: 1786211403034443 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:50:03.245201 31636 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:50:03.246115 31926 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [term 1 FOLLOWER]: Refusing update from remote peer d6391e6017354812a7c36c0732013ad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:50:03.246829 32021 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:50:03.248520 31976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6391e6017354812a7c36c0732013ad0. Latest consensus state: current_term: 1 leader_uuid: "d6391e6017354812a7c36c0732013ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.248634 31976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:03.250126 32021 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6391e6017354812a7c36c0732013ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.250226 32021 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:50:03.250692 31976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6391e6017354812a7c36c0732013ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.250787 31976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:03.251004 32034 mvcc.cc:205] Tried to move back new op lower bound from 7316321907693457408 to 7316321907681853440. Current Snapshot: MvccSnapshot[applied={T|T < 7316321907693457408}]
I20260808 17:50:03.251608 32036 mvcc.cc:205] Tried to move back new op lower bound from 7316321907693457408 to 7316321907681853440. Current Snapshot: MvccSnapshot[applied={T|T < 7316321907693457408}]
I20260808 17:50:03.252092 32031 catalog_manager.cc:1347] Generated new cluster ID: 1c64824371994c98bea75ec4c062c153
I20260808 17:50:03.252144 32031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:50:03.254750 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.242424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.29.62.253:42873)
0808 17:50:03.242463 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.242465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.246534 (+ 4069us) server_negotiation.cc:407] Connection header received
0808 17:50:03.246603 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.246605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.246618 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.246666 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.247450 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.248138 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.249158 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.249465 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.249508 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.251657 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.251658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.251659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.251666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.253441 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.253867 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.253868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.253869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.253929 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.254129 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.254130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.254130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.254257 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.254326 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.254570 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.254653 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:50:03.255896 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.242117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47276 (local address 127.29.62.254:39349)
0808 17:50:03.242158 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.242160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.243228 (+ 1068us) server_negotiation.cc:407] Connection header received
0808 17:50:03.243338 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.243339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.243351 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.243411 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.244264 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.245005 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.246053 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.246344 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.246448 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.250482 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.250484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.250484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.250492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.253935 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.254550 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.254551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.254564 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.254616 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.255219 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.255220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.255220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.255354 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.255444 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.255738 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.255810 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:50:03.262477 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.242571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33248 (local address 127.29.62.252:37627)
0808 17:50:03.242609 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.242611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.255414 (+ 12803us) server_negotiation.cc:407] Connection header received
0808 17:50:03.255515 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.255516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.255538 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.255567 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.256120 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.256884 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.257882 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.258218 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.258283 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.259628 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.259629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.259629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.259636 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.261278 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.261659 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.261660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.261661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.261726 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.261941 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.261942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.261943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.262069 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.262185 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.262291 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.262391 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:50:03.268352 32031 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:50:03.268831 31854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } attempt: 3
I20260808 17:50:03.269083 32031 catalog_manager.cc:1504] Loading token signing keys...
Aug 08 17:50:03 dist-test-slave-tk9z krb5kdc[31619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211403, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
I20260808 17:50:03.277695 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.269070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37627 (local address 127.0.0.1:33252)
0808 17:50:03.269171 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:03.269183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:03.269228 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:03.269395 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:03.269396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:03.269576 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:03.269622 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.269624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.270516 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.270518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.271289 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:03.271292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.271814 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.271815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:03.271881 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.272386 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:03.272391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:03.273870 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:03.276453 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.276455 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.276455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.276664 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.276934 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:03.276935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:03.276935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:03.276998 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:03.277385 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:03.277385 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:03.277448 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 17:50:03.277502 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:50:03.277596 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 17:50:03.278493 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.269160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33252 (local address 127.29.62.252:37627)
0808 17:50:03.269198 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.269200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.269211 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:50:03.269290 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.269291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.269301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.269349 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.269699 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.270448 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.271396 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.271701 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.271750 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.273970 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:03.273972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:03.273972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:03.273979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:03.276314 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.276752 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.276753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.276754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.276860 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:03.277072 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:03.277073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:03.277073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:03.277306 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:03.278108 (+ 802us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.278180 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.278312 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:50:03.280196 31854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74: Bootstrap starting.
I20260808 17:50:03.280987 31854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74: Neither blocks nor log segments found. Creating new log.
I20260808 17:50:03.281270 31854 log.cc:824] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74: Log is configured to *not* fsync() on all Append() calls
I20260808 17:50:03.286747 31854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74: No bootstrap required, opened a new log
I20260808 17:50:03.288856 31854 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } }
I20260808 17:50:03.289016 31854 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2dadfa4e5a449cc97000bb0b824ed74, State: Initialized, Role: FOLLOWER
I20260808 17:50:03.289104 31854 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } }
I20260808 17:50:03.289467 31854 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:50:03.290050 31810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:50:03.291091 31810 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [term 1 FOLLOWER]: Refusing update from remote peer d6391e6017354812a7c36c0732013ad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:50:03.291373 32021 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:50:03.291389 31811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6391e6017354812a7c36c0732013ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2dadfa4e5a449cc97000bb0b824ed74"
I20260808 17:50:03.292155 31811 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6391e6017354812a7c36c0732013ad0 in term 1.
I20260808 17:50:03.292613 32031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0: Generated new TSK 0
I20260808 17:50:03.292815 32031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:50:03.294480 32052 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6391e6017354812a7c36c0732013ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.294603 32052 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:03.294665 32052 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6391e6017354812a7c36c0732013ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.294719 32052 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:03.295202 32075 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6391e6017354812a7c36c0732013ad0. Latest consensus state: current_term: 1 leader_uuid: "d6391e6017354812a7c36c0732013ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.295288 32075 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:03.295965 32082 catalog_manager.cc:1259] Loaded cluster ID: 1c64824371994c98bea75ec4c062c153
I20260808 17:50:03.296020 32082 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74: loading cluster ID for follower catalog manager: success
I20260808 17:50:03.297142 32082 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74: acquiring CA information for follower catalog manager: success
I20260808 17:50:03.297478 32082 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2dadfa4e5a449cc97000bb0b824ed74: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 17:50:03.325737 31862 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.323949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.29.62.253:42873)
0808 17:50:03.324002 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.324005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.324948 (+ 943us) server_negotiation.cc:407] Connection header received
0808 17:50:03.325088 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.325090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.325101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.325133 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.325663 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60522: BlockingRecv error: recv got EOF from 127.0.0.1:60522 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 17:50:03.326567 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.324361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33262 (local address 127.29.62.252:37627)
0808 17:50:03.324403 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.324405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.326008 (+ 1603us) server_negotiation.cc:407] Connection header received
0808 17:50:03.326108 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.326109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.326120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.326149 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.326506 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33262: BlockingRecv error: recv got EOF from 127.0.0.1:33262 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:50:03.327320 31741 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.324688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47278 (local address 127.29.62.254:39349)
0808 17:50:03.324729 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.324731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.326768 (+ 2037us) server_negotiation.cc:407] Connection header received
0808 17:50:03.326869 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.326870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.326881 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.326910 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.327241 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47278: BlockingRecv error: recv got EOF from 127.0.0.1:47278 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:50:03.332628 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.329012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.29.62.253:42873)
0808 17:50:03.329048 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.329050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.329254 (+ 204us) server_negotiation.cc:407] Connection header received
0808 17:50:03.329378 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.329379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.329425 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.329471 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.329662 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.330495 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.331660 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.331948 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.331989 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.332212 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.332361 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.332416 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.332533 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.332553 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:50:03.335918 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.335099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33278 (local address 127.29.62.252:37627)
0808 17:50:03.335136 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.335137 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.335364 (+ 227us) server_negotiation.cc:407] Connection header received
0808 17:50:03.335478 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.335478 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.335498 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.335529 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.335868 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33278: BlockingRecv error: recv got EOF from 127.0.0.1:33278 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:50:03.341202 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.338078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.29.62.253:42873)
0808 17:50:03.338112 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.338113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.338442 (+ 329us) server_negotiation.cc:407] Connection header received
0808 17:50:03.338535 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.338536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.338545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.338572 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.338740 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.339447 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.340441 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.340717 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.340758 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.340946 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.340981 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.341012 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.341110 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.341130 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:50:03.344508 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.343612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.29.62.252:37627)
0808 17:50:03.343649 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.343651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.343861 (+ 210us) server_negotiation.cc:407] Connection header received
0808 17:50:03.343967 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.343967 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.343978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.344007 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.344459 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33280: BlockingRecv error: recv got EOF from 127.0.0.1:33280 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:50:03.351071 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.347649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.29.62.253:42873)
0808 17:50:03.347689 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.347690 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.347948 (+ 258us) server_negotiation.cc:407] Connection header received
0808 17:50:03.348054 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.348055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.348066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.348094 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.348281 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.349037 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.350202 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.350528 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.350576 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.350786 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.350821 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.350851 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.350970 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.350991 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:50:03.354349 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.353521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33288 (local address 127.29.62.252:37627)
0808 17:50:03.353563 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.353565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.353780 (+ 215us) server_negotiation.cc:407] Connection header received
0808 17:50:03.353883 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.353884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.353897 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.353925 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.354282 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33288: BlockingRecv error: recv got EOF from 127.0.0.1:33288 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:50:03.362989 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.359608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.29.62.253:42873)
0808 17:50:03.359651 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.359653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.359927 (+ 274us) server_negotiation.cc:407] Connection header received
0808 17:50:03.360032 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.360033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.360044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.360074 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.360271 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.361003 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.362115 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.362452 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.362501 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.362710 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.362745 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.362774 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.362886 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.362907 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:50:03.366308 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.365417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.29.62.252:37627)
0808 17:50:03.365463 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.365464 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.365690 (+ 226us) server_negotiation.cc:407] Connection header received
0808 17:50:03.365795 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.365796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.365809 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.365840 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.366233 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33290: BlockingRecv error: recv got EOF from 127.0.0.1:33290 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:50:03.379254 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.375678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.29.62.253:42873)
0808 17:50:03.375724 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.375726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.376031 (+ 305us) server_negotiation.cc:407] Connection header received
0808 17:50:03.376141 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.376142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.376153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.376183 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.376402 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.377152 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.378325 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.378687 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.378737 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.378966 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.379001 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.379031 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.379142 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.379165 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 17:50:03.382890 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.381895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33292 (local address 127.29.62.252:37627)
0808 17:50:03.381936 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.381938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.382203 (+ 265us) server_negotiation.cc:407] Connection header received
0808 17:50:03.382319 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.382320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.382333 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.382364 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.382827 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33292: BlockingRecv error: recv got EOF from 127.0.0.1:33292 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:50:03.403928 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.400298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.29.62.253:42873)
0808 17:50:03.400344 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.400346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.400685 (+ 339us) server_negotiation.cc:407] Connection header received
0808 17:50:03.400801 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.400802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.400813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.400845 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.401079 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.401829 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.403069 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.403372 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.403422 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.403645 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.403682 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.403712 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.403822 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.403844 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 17:50:03.407320 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.406425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.29.62.252:37627)
0808 17:50:03.406467 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.406469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.406713 (+ 244us) server_negotiation.cc:407] Connection header received
0808 17:50:03.406814 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.406815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.406828 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.406858 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.407244 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33304: BlockingRecv error: recv got EOF from 127.0.0.1:33304 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:50:03.443466 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.440851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.29.62.253:42873)
0808 17:50:03.440905 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.440907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.441062 (+ 155us) server_negotiation.cc:407] Connection header received
0808 17:50:03.441142 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.441143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.441161 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.441226 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.441387 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.441872 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.442736 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.442930 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.443026 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.443188 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.443225 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.443277 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.443348 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.443368 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 17:50:03.446664 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.445623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.29.62.252:37627)
0808 17:50:03.445663 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.445666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.446087 (+ 421us) server_negotiation.cc:407] Connection header received
0808 17:50:03.446153 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.446154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.446168 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.446227 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.446611 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33308: BlockingRecv error: recv got EOF from 127.0.0.1:33308 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:50:03.514269 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.512052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60592 (local address 127.29.62.253:42873)
0808 17:50:03.512147 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.512149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.512206 (+ 57us) server_negotiation.cc:407] Connection header received
0808 17:50:03.512273 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.512274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.512283 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.512327 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.512457 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.512938 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.513657 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.513833 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.513882 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.514017 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.514042 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.514080 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.514145 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.514161 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 17:50:03.516501 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.515949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33322 (local address 127.29.62.252:37627)
0808 17:50:03.516022 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.516023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.516057 (+ 34us) server_negotiation.cc:407] Connection header received
0808 17:50:03.516112 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.516113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.516123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.516169 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.516433 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33322: BlockingRecv error: recv got EOF from 127.0.0.1:33322 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:50:03.648116 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.645714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.29.62.253:42873)
0808 17:50:03.645782 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.645784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.645889 (+ 105us) server_negotiation.cc:407] Connection header received
0808 17:50:03.645939 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.645940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.645950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.646016 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.646156 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.646687 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.647467 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.647649 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.647705 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.647859 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.647886 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.647923 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.647970 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.647986 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 17:50:03.650614 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.649922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.29.62.252:37627)
0808 17:50:03.650013 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.650015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.650048 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:50:03.650104 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.650105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.650115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.650157 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.650538 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33328: BlockingRecv error: recv got EOF from 127.0.0.1:33328 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 17:50:03.864060 31993 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6391e6017354812a7c36c0732013ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6391e6017354812a7c36c0732013ad0" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 39349 } } peers { permanent_uuid: "c2dadfa4e5a449cc97000bb0b824ed74" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 42873 } } peers { permanent_uuid: "53d89d265e2642eca295a79117a5801c" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37627 } } }
I20260808 17:50:03.864259 31993 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6391e6017354812a7c36c0732013ad0 [sys.catalog]: This master's current role is: LEADER
I20260808 17:50:03.910192 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:03.907888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.29.62.253:42873)
0808 17:50:03.907956 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.907958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.908026 (+ 68us) server_negotiation.cc:407] Connection header received
0808 17:50:03.908092 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.908093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.908103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.908146 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:03.908326 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.908828 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.909578 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:03.909759 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:03.909813 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:03.909941 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:03.909967 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:03.910006 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:03.910060 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 17:50:03.910076 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 17:50:03.912621 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:03.912024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33340 (local address 127.29.62.252:37627)
0808 17:50:03.912075 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:03.912077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:03.912086 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:50:03.912146 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:03.912147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:03.912156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:03.912209 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:03.912561 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33340: BlockingRecv error: recv got EOF from 127.0.0.1:33340 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:50:04.237179 31990 catalog_manager.cc:1259] Loaded cluster ID: 1c64824371994c98bea75ec4c062c153
I20260808 17:50:04.237251 31990 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c: loading cluster ID for follower catalog manager: success
I20260808 17:50:04.238029 31990 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c: acquiring CA information for follower catalog manager: success
I20260808 17:50:04.238284 31990 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 53d89d265e2642eca295a79117a5801c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:50:04.428534 32273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:04.425822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.29.62.253:42873)
0808 17:50:04.425966 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:04.425968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:04.426060 (+ 92us) server_negotiation.cc:407] Connection header received
0808 17:50:04.426169 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:04.426170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:04.426183 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:04.426265 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:04.426473 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.426991 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.427783 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.428010 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.428090 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:04.428238 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:04.428279 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:04.428343 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:04.428404 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 17:50:04.428434 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260808 17:50:04.432753 32284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:04.430492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.29.62.252:37627)
0808 17:50:04.430624 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:04.430626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:04.430634 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:50:04.430657 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:04.430657 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:04.430708 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:04.430780 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:04.430900 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.431365 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.432085 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.432295 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.432360 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:04.432470 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:04.432580 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:04.432635 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:04.432668 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 17:50:04.432687 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260808 17:50:04.437763 32293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:04.434644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47290 (local address 127.29.62.254:39349)
0808 17:50:04.434766 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:04.434768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:04.434937 (+ 169us) server_negotiation.cc:407] Connection header received
0808 17:50:04.435134 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:04.435135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:04.435194 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:04.435279 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:04.435401 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.435893 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.436818 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.437040 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.437128 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:04.437324 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:04.437419 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:04.437509 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:04.437568 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 17:50:04.437592 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260808 17:50:04.438027 32273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:04.434609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.29.62.253:42873)
0808 17:50:04.434655 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:04.434656 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:04.434798 (+ 142us) server_negotiation.cc:407] Connection header received
0808 17:50:04.434917 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:04.434918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:04.434930 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:04.434995 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:04.435175 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.435939 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.437157 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.437452 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.437493 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:04.437703 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:04.437734 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:04.437761 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:04.437906 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 17:50:04.437930 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:50:04.438107 32284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:04.434803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.29.62.252:37627)
0808 17:50:04.434855 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:04.434857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:04.434973 (+ 116us) server_negotiation.cc:407] Connection header received
0808 17:50:04.435100 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:04.435102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:04.435112 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:04.435139 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:04.435323 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.436017 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.437157 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:04.437455 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:04.437493 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:04.437717 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:04.437751 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:04.437806 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:04.438010 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 17:50:04.438036 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:50:04.439847 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31636
I20260808 17:50:04.458997 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31749
I20260808 17:50:04.473213 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 31865
2026-08-08T17:50:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3101 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:50:04 dist-test-slave-tk9z krb5kdc[32299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:50:04 dist-test-slave-tk9z krb5kdc[32299](info): set up 2 sockets
Aug 08 17:50:04 dist-test-slave-tk9z krb5kdc[32299](info): commencing operation
krb5kdc: starting...
W20260808 17:50:05.751768 29947 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/test-user.keytab.
Aug 08 17:50:05 dist-test-slave-tk9z krb5kdc[32299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:50:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:50:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.62.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
Entry for principal kudu/127.29.62.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab.
I20260808 17:50:05.880631 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34075
--webserver_interface=127.29.62.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:34075,127.29.62.253:41179,127.29.62.252:37601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:50:05.965481 32320 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:50:05.965746 32320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:50:05.965781 32320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:50:05.965812 32320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:50:05.966933 32320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:50:05.966976 32320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:50:05.966991 32320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:50:05.967005 32320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:50:05.967017 32320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:50:05.968461 32320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:34075,127.29.62.253:41179,127.29.62.252:37601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.254.keytab
--principal=kudu/127.29.62.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.254:34075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.62.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:50:05.968664 32320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:50:05.968835 32320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:50:05.973835 32334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:05.973841 32330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:05.973851 32331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:05.974077 32320 server_base.cc:1034] running on GCE node
I20260808 17:50:05.974411 32320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:50:05.974722 32320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:50:05.975886 32320 hybrid_clock.cc:648] HybridClock initialized: now 1786211405975871 us; error 29 us; skew 500 ppm
Aug 08 17:50:05 dist-test-slave-tk9z krb5kdc[32299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:50:05.979159 32320 init.cc:377] Logged in from keytab as kudu/127.29.62.254@KRBTEST.COM (short username kudu)
I20260808 17:50:05.979794 32320 webserver.cc:460] Webserver started at http://127.29.62.254:40065/ using document root <none> and password file <none>
I20260808 17:50:05.980088 32320 fs_manager.cc:359] Metadata directory not provided
I20260808 17:50:05.980154 32320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:50:05.980293 32320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:50:05.981401 32320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/data/instance:
uuid: "1ca74876bd69418bb1e253dce37a9092"
format_stamp: "Formatted at 2026-08-08 17:50:05 on dist-test-slave-tk9z"
server_key: "54f26af3dd687682f04d5b315d90073b"
server_key_iv: "50a648a168df2f7b9551d7dace556db9"
server_key_version: "encryptionkey@0"
I20260808 17:50:05.981804 32320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/wal/instance:
uuid: "1ca74876bd69418bb1e253dce37a9092"
format_stamp: "Formatted at 2026-08-08 17:50:05 on dist-test-slave-tk9z"
server_key: "54f26af3dd687682f04d5b315d90073b"
server_key_iv: "50a648a168df2f7b9551d7dace556db9"
server_key_version: "encryptionkey@0"
I20260808 17:50:05.983304 32320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:50:05.984066 32343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:50:05.984226 32320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:50:05.984337 32320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/wal
uuid: "1ca74876bd69418bb1e253dce37a9092"
format_stamp: "Formatted at 2026-08-08 17:50:05 on dist-test-slave-tk9z"
server_key: "54f26af3dd687682f04d5b315d90073b"
server_key_iv: "50a648a168df2f7b9551d7dace556db9"
server_key_version: "encryptionkey@0"
I20260808 17:50:05.984416 32320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:50:05.995846 32320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:50:05.997687 32320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:50:05.997857 32320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:50:06.002494 32320 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.254:34075
I20260808 17:50:06.002544 32427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.254:34075 every 8 connection(s)
I20260808 17:50:06.002965 32320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/data/info.pb
I20260808 17:50:06.004067 32429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:50:06.005626 32429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:06.006503 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 32320
I20260808 17:50:06.006651 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-0/wal/instance
I20260808 17:50:06.006870 29947 external_mini_cluster.cc:1428] Setting key 7ed840d9f7425ca8da67711b77ba2d11
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.62.253@KRBTEST.COM; defaulting to no policy
I20260808 17:50:06.015928 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.006356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.29.62.254:34075)
0808 17:50:06.006580 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.006591 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.006738 (+ 147us) server_negotiation.cc:407] Connection header received
0808 17:50:06.007379 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.007382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.007409 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.007510 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.008099 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.008946 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.009837 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.010147 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.010296 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.012493 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.012500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.012501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.012508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.014173 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.014664 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.014667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.014668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.014755 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.015002 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.015003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.015003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.015219 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.015317 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.015526 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.015740 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":177,"thread_start_us":137,"threads_started":1}
I20260808 17:50:06.015928 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.006322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34075 (local address 127.0.0.1:33210)
0808 17:50:06.006597 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.006637 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.007207 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.007516 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.007518 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.007821 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.007986 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.007991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.009051 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.009052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.009736 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.009739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.010271 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.010272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.010348 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.010954 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.010963 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.012383 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.014280 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.014287 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.014289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.014568 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.014832 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.014833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.014834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.014928 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.015301 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.015307 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.015373 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.015476 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.015669 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"spinlock_wait_cycles":9600,"thread_start_us":189,"threads_started":1}
I20260808 17:50:06.016770 32429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:06.017141 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.017012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33772)
0808 17:50:06.017064 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.017113 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 17:50:06.017369 32347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
I20260808 17:50:06.017906 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.017782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33776)
0808 17:50:06.017848 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.017880 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.253@KRBTEST.COM" created.
W20260808 17:50:06.018085 32429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:41179: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab.
Entry for principal kudu/127.29.62.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab.
I20260808 17:50:06.032997 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.253:41179
--webserver_interface=127.29.62.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab
--principal=kudu/127.29.62.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:34075,127.29.62.253:41179,127.29.62.252:37601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:50:06.067355 32429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } attempt: 1
I20260808 17:50:06.067930 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.067753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33784)
0808 17:50:06.067832 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.067886 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 17:50:06.068346 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.068250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33800)
0808 17:50:06.068297 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.068326 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 17:50:06.068449 32429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:41179: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
W20260808 17:50:06.116674 32440 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:50:06.116880 32440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:50:06.116897 32440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:50:06.116916 32440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:50:06.117961 32440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:50:06.117995 32440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:50:06.118008 32440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:50:06.118021 32440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:50:06.118032 32440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:50:06.119448 32440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:34075,127.29.62.253:41179,127.29.62.252:37601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.253.keytab
--principal=kudu/127.29.62.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.253:41179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.62.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:50:06.119637 32440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:50:06.119802 32440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:50:06.124501 32448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:06.124478 32452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:06.124640 32440 server_base.cc:1034] running on GCE node
W20260808 17:50:06.124725 32450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:06.125023 32440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:50:06.125308 32440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:50:06.126446 32440 hybrid_clock.cc:648] HybridClock initialized: now 1786211406126434 us; error 25 us; skew 500 ppm
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:50:06.128895 32440 init.cc:377] Logged in from keytab as kudu/127.29.62.253@KRBTEST.COM (short username kudu)
I20260808 17:50:06.129318 32440 webserver.cc:460] Webserver started at http://127.29.62.253:42889/ using document root <none> and password file <none>
I20260808 17:50:06.129550 32440 fs_manager.cc:359] Metadata directory not provided
I20260808 17:50:06.129619 32440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:50:06.129773 32440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:50:06.130927 32440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/data/instance:
uuid: "3ae46f67e2224efd855e88e7cc33077d"
format_stamp: "Formatted at 2026-08-08 17:50:06 on dist-test-slave-tk9z"
server_key: "1e3f37b1f9c5118c0d4334324d0b389d"
server_key_iv: "8086406f296f539f64f69fe19473ff6b"
server_key_version: "encryptionkey@0"
I20260808 17:50:06.131306 32440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/wal/instance:
uuid: "3ae46f67e2224efd855e88e7cc33077d"
format_stamp: "Formatted at 2026-08-08 17:50:06 on dist-test-slave-tk9z"
server_key: "1e3f37b1f9c5118c0d4334324d0b389d"
server_key_iv: "8086406f296f539f64f69fe19473ff6b"
server_key_version: "encryptionkey@0"
I20260808 17:50:06.132741 32440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:50:06.133553 32464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:50:06.133749 32440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:50:06.133862 32440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/wal
uuid: "3ae46f67e2224efd855e88e7cc33077d"
format_stamp: "Formatted at 2026-08-08 17:50:06 on dist-test-slave-tk9z"
server_key: "1e3f37b1f9c5118c0d4334324d0b389d"
server_key_iv: "8086406f296f539f64f69fe19473ff6b"
server_key_version: "encryptionkey@0"
I20260808 17:50:06.133939 32440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:50:06.136603 32429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } attempt: 2
I20260808 17:50:06.137010 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.136871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33808)
0808 17:50:06.136945 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.136980 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 17:50:06.137396 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.137298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33820)
0808 17:50:06.137335 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.137370 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 17:50:06.137490 32429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.253:41179: Network error: Client connection negotiation failed: client connection to 127.29.62.253:41179: connect: Connection refused (error 111)
I20260808 17:50:06.143661 32440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:50:06.145552 32440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:50:06.145714 32440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:50:06.149834 32440 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.253:41179
I20260808 17:50:06.149866 32541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.253:41179 every 8 connection(s)
I20260808 17:50:06.150267 32440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/data/info.pb
I20260808 17:50:06.151368 32542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:50:06.152963 32542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:06.158550 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 32440
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:50:06.158641 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-1/wal/instance
I20260808 17:50:06.158859 29947 external_mini_cluster.cc:1428] Setting key 34151d9bd3ef3ba627691e18672112b7
I20260808 17:50:06.161866 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.153679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.29.62.254:34075)
0808 17:50:06.153729 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.153731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.153943 (+ 212us) server_negotiation.cc:407] Connection header received
0808 17:50:06.154488 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.154490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.154503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.154548 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.155185 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.155660 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.156635 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.156837 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.156921 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.159126 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.159127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.159128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.159136 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.160477 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.160894 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.160895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.160896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.160963 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.161211 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.161212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.161213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.161331 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.161404 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.161686 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.161778 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:50:06.162034 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.153634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34075 (local address 127.0.0.1:33212)
0808 17:50:06.153814 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.153849 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.154342 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.154629 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.154633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.154892 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.155084 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.155087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.155757 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.155758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.156544 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.156546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.156952 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.156954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.157070 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.157576 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.157585 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.159036 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.160575 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.160582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.160584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.160804 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.161032 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.161033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.161033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.161144 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.161413 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.161420 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.161552 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.161652 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.161805 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260808 17:50:06.162544 32542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.62.252@KRBTEST.COM; defaulting to no policy
I20260808 17:50:06.170192 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.162821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33832)
0808 17:50:06.162870 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.162880 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.162921 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.163156 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.163157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.163319 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.163376 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.163378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.164794 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.164795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.165269 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.165270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.165947 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.165947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.166001 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.166453 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.166458 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.167568 (+ 1110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.169205 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.169205 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.169206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.169367 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.169578 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.169579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.169579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.169630 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.169882 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.169883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.169989 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.170048 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.170124 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 17:50:06.170397 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.162864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.29.62.253:41179)
0808 17:50:06.163015 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.163019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.163039 (+ 20us) server_negotiation.cc:407] Connection header received
0808 17:50:06.163065 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.163066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.163076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.163157 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.163515 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.164693 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.165352 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.165864 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.165943 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.167653 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.167662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.167663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.167670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.169132 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.169446 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.169449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.169449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.169519 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.169684 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.169685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.169685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.169822 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.169885 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.170112 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.170202 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.62.252@KRBTEST.COM" created.
I20260808 17:50:06.170997 32542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:06.171386 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.171266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44072)
0808 17:50:06.171320 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.171366 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 17:50:06.171545 32465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
I20260808 17:50:06.171936 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.171872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44080)
0808 17:50:06.171896 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.171920 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 17:50:06.172102 32542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37601: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.62.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab.
Entry for principal kudu/127.29.62.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab.
I20260808 17:50:06.185577 29947 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGk3gfK/build/release/bin/kudu
/tmp/dist-test-taskGk3gfK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.252:37601
--webserver_interface=127.29.62.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab
--principal=kudu/127.29.62.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.62.212:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.62.254:34075,127.29.62.253:41179,127.29.62.252:37601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:50:06.221331 32542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } attempt: 1
I20260808 17:50:06.221854 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.221712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44084)
0808 17:50:06.221787 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.221824 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 17:50:06.222324 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.222185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44098)
0808 17:50:06.222250 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.222291 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 17:50:06.222474 32542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37601: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
I20260808 17:50:06.228629 32429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } attempt: 3
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
I20260808 17:50:06.236451 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.228949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33842)
0808 17:50:06.229033 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.229045 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.229095 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.229299 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.229301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.229510 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.229592 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.229594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.230271 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.230272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.231130 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.231133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.231498 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.231499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.231587 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.232085 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.232091 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.233527 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.235332 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.235333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.235333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.235528 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.235763 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.235764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.235764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.235822 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.236138 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.236139 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.236201 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.236291 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.236377 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 17:50:06.236676 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.229019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33842 (local address 127.29.62.253:41179)
0808 17:50:06.229161 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.229162 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.229170 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:50:06.229193 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.229194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.229204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.229270 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.229687 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.230177 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.231212 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.231424 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.231483 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.233639 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.233641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.233641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.233650 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.235235 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.235610 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.235612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.235612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.235686 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.235885 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.235886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.235887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.236031 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.236131 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.236326 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.236421 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260808 17:50:06.237069 32429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:06.237457 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.237357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44100)
0808 17:50:06.237406 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.237438 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 17:50:06.237843 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.237744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44112)
0808 17:50:06.237791 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.237815 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 17:50:06.237957 32429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37601: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
I20260808 17:50:06.269112 32429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } attempt: 1
I20260808 17:50:06.269680 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.269533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44126)
0808 17:50:06.269607 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.269649 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 17:50:06.270077 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.269996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44132)
0808 17:50:06.270037 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.270060 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 17:50:06.270202 32429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37601: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
W20260808 17:50:06.276821 32553 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:50:06.277043 32553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:50:06.277060 32553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:50:06.277125 32553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:50:06.278270 32553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:50:06.278312 32553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:50:06.278326 32553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:50:06.278365 32553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:50:06.278416 32553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:50:06.279985 32553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.62.212:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.62.254:34075,127.29.62.253:41179,127.29.62.252:37601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/krb5kdc/kudu_127.29.62.252.keytab
--principal=kudu/127.29.62.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.62.252:37601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.62.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:03:43 UTC on ccycloud.quasar-rzkrkr.root.comops.site
build id 4574927
I20260808 17:50:06.280254 32553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:50:06.280515 32553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:50:06.285432 32563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:06.285498 32559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:50:06.285480 32558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:50:06.285681 32553 server_base.cc:1034] running on GCE node
I20260808 17:50:06.286031 32553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:50:06.286464 32553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:50:06.287639 32553 hybrid_clock.cc:648] HybridClock initialized: now 1786211406287611 us; error 52 us; skew 500 ppm
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:50:06.290194 32553 init.cc:377] Logged in from keytab as kudu/127.29.62.252@KRBTEST.COM (short username kudu)
I20260808 17:50:06.290640 32542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } attempt: 2
I20260808 17:50:06.290747 32553 webserver.cc:460] Webserver started at http://127.29.62.252:38299/ using document root <none> and password file <none>
I20260808 17:50:06.290999 32553 fs_manager.cc:359] Metadata directory not provided
I20260808 17:50:06.291051 32553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:50:06.291078 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.290907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44140)
0808 17:50:06.290980 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.291019 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 17:50:06.291199 32553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:50:06.291520 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.291430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44154)
0808 17:50:06.291472 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.291500 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 17:50:06.291657 32542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.62.252:37601: Network error: Client connection negotiation failed: client connection to 127.29.62.252:37601: connect: Connection refused (error 111)
I20260808 17:50:06.292412 32553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/data/instance:
uuid: "c90f19bd04294c5da80a6fc5328f428e"
format_stamp: "Formatted at 2026-08-08 17:50:06 on dist-test-slave-tk9z"
server_key: "012b9a4eac286a576f95799f8a96c660"
server_key_iv: "63b61466f63fbb94cbdd4c2d0cc420a2"
server_key_version: "encryptionkey@0"
I20260808 17:50:06.292732 32553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/wal/instance:
uuid: "c90f19bd04294c5da80a6fc5328f428e"
format_stamp: "Formatted at 2026-08-08 17:50:06 on dist-test-slave-tk9z"
server_key: "012b9a4eac286a576f95799f8a96c660"
server_key_iv: "63b61466f63fbb94cbdd4c2d0cc420a2"
server_key_version: "encryptionkey@0"
I20260808 17:50:06.294149 32553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:50:06.295019 32573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:50:06.295269 32553 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:50:06.295348 32553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/data,/tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/wal
uuid: "c90f19bd04294c5da80a6fc5328f428e"
format_stamp: "Formatted at 2026-08-08 17:50:06 on dist-test-slave-tk9z"
server_key: "012b9a4eac286a576f95799f8a96c660"
server_key_iv: "63b61466f63fbb94cbdd4c2d0cc420a2"
server_key_version: "encryptionkey@0"
I20260808 17:50:06.295403 32553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:50:06.325945 32553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:50:06.328430 32553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:50:06.328553 32553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:50:06.332453 32553 rpc_server.cc:307] RPC server started. Bound to: 127.29.62.252:37601
I20260808 17:50:06.332530 32654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.62.252:37601 every 8 connection(s)
I20260808 17:50:06.332890 32553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/data/info.pb
I20260808 17:50:06.333530 29947 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGk3gfK/build/release/bin/kudu as pid 32553
I20260808 17:50:06.333654 29947 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGk3gfK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786211125631729-29947-0/minicluster-data/master-2/wal/instance
I20260808 17:50:06.333875 29947 external_mini_cluster.cc:1428] Setting key 2b01b0648602407d45bf53b5a0bcec4a
I20260808 17:50:06.334061 32655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:50:06.336565 32655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:50:06.344815 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.337316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.29.62.254:34075)
0808 17:50:06.337369 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.337371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.337490 (+ 119us) server_negotiation.cc:407] Connection header received
0808 17:50:06.337547 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.337548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.337560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.337624 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.338024 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.338692 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.339562 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.339756 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.339828 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.342034 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.342035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.342036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.342043 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.343565 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.343967 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.343968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.343968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.344020 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.344217 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.344217 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.344217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.344353 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.344416 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.344686 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.344750 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:50:06.344828 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.337205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.254:34075 (local address 127.0.0.1:33228)
0808 17:50:06.337370 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.337413 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.337486 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.337647 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.337650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.337876 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.337947 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.337949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.338767 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.338768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.339473 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.339475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.339875 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.339876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.339946 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.340392 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.340400 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.341946 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.343641 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.343646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.343649 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.343888 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.344084 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.344084 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.344085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.344151 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.344420 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.344425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.344469 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.344560 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.344645 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":113,"threads_started":1}
I20260808 17:50:06.344828 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.334403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.29.62.252:37601)
0808 17:50:06.334608 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.334618 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.334653 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:50:06.335330 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.335339 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.335413 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.335570 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.336176 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.337466 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.338508 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.339021 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.339130 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.341547 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.341555 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.341556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.341563 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.343132 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.343491 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.343494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.343494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.343578 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.343820 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.343821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.343821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.344121 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.344264 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.344453 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.344590 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260808 17:50:06.345359 32429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } attempt: 2
I20260808 17:50:06.345408 32655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211405, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.254@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.254@KRBTEST.COM
I20260808 17:50:06.353622 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.345697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.253:41179 (local address 127.0.0.1:33850)
0808 17:50:06.345753 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.345762 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.345797 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.345969 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.345979 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.346125 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.346170 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.346171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.347039 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.347040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.347497 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.347498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.347884 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.347885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.347957 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.348449 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.348454 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.350651 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.352563 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.352564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.352564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.352769 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.352995 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.352996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.352996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.353055 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.353315 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.353316 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.353357 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.353436 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.353529 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 17:50:06.353647 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.345760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33850 (local address 127.29.62.253:41179)
0808 17:50:06.345815 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.345818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.345827 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:50:06.345853 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.345854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.345865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.345916 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.346233 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.346957 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.347583 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.347813 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.347875 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.350742 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.350744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.350744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.350753 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.352481 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.352849 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.352851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.352851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.352911 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.353110 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.353112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.353112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.353233 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.353309 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.353482 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.353557 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:50:06.353973 32655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:50:06.354107 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.345614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44168)
0808 17:50:06.345655 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.345667 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.345706 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.345894 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.345895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.346058 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.346099 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.346101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.347013 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.347013 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.347754 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.347756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.348149 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.348150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.348216 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.348866 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.348869 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.350538 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.352498 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.352499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.352499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.352813 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.353022 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.353024 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.353024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.353087 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.353382 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.353383 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.353442 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.353493 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.353583 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 17:50:06.354569 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.345720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44168 (local address 127.29.62.252:37601)
0808 17:50:06.345803 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.345805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.345812 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:50:06.345830 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.345831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.345840 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.345889 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.346176 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.346933 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.347830 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.348074 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.348147 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.350634 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.350635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.350635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.350642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.352228 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.352889 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.352890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.352890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.352943 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.353166 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.353167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.353167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.353299 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.354292 (+ 993us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.354333 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.354465 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 17:50:06.354790 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.345983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33234 (local address 127.29.62.254:34075)
0808 17:50:06.346019 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.346021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.346304 (+ 283us) server_negotiation.cc:407] Connection header received
0808 17:50:06.346363 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.346364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.346395 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.346449 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.346899 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.347615 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.348564 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.348767 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.348818 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.351546 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.351547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.351547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.351553 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.353127 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.353496 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.353497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.353498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.353548 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.353753 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.353754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.353754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.354193 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.354363 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.354577 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.354674 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:50:06.356907 32429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092: Bootstrap starting.
I20260808 17:50:06.357700 32429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092: Neither blocks nor log segments found. Creating new log.
I20260808 17:50:06.358049 32429 log.cc:824] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092: Log is configured to *not* fsync() on all Append() calls
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.252@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
I20260808 17:50:06.359022 32429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092: No bootstrap required, opened a new log
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.62.253@KRBTEST.COM
I20260808 17:50:06.360911 32429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } }
I20260808 17:50:06.361020 32429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca74876bd69418bb1e253dce37a9092, State: Initialized, Role: FOLLOWER
I20260808 17:50:06.361104 32429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } }
I20260808 17:50:06.361434 32429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:50:06.361392 32669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.361487 32669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:06.362339 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.354250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44170)
0808 17:50:06.354353 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.354365 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.354437 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.354763 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.354764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.354903 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.354935 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.354937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.355750 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.355751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.356207 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.356208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.357045 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.357046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.357094 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.357498 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.357503 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.358962 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.361264 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.361265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.361265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.361481 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.361670 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.361671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.361671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.361718 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.362038 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.362039 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.362073 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.362139 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.362230 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 17:50:06.362646 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.354310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.29.62.252:37601)
0808 17:50:06.354640 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.354642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.354649 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:50:06.354668 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.354669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.354678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.354729 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.355008 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.355679 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.356799 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.356970 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.357049 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.359060 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.359062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.359062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.359070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.361171 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.361563 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.361564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.361564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.361612 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.361775 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.361775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.361776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.361940 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.362015 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.362460 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.362539 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":134,"threads_started":1}
I20260808 17:50:06.363389 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.355512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33864 (local address 127.29.62.253:41179)
0808 17:50:06.355546 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.355548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.355555 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:50:06.355618 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.355618 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.355627 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.355680 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.356284 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.356719 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.357599 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.357917 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.357991 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.360336 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.360338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.360338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.360345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.362053 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.362497 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.362498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.362498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.362553 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.362864 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.362864 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.362865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.363003 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.363174 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.363214 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.363286 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:50:06.364176 32655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e: Bootstrap starting.
I20260808 17:50:06.364626 29947 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:50:06.365147 32655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e: Neither blocks nor log segments found. Creating new log.
I20260808 17:50:06.365435 32655 log.cc:824] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e: Log is configured to *not* fsync() on all Append() calls
W20260808 17:50:06.363350 32687 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:50:06.366195 32687 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:50:06.365381 32381 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1ca74876bd69418bb1e253dce37a9092"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33234
I20260808 17:50:06.366269 32655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e: No bootstrap required, opened a new log
I20260808 17:50:06.366281 32381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 17:50:06.366328 32381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:50:06.367249 32381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } }
I20260808 17:50:06.367544 32381 leader_election.cc:288] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [CANDIDATE]: Term 1 election: Requested vote from peers 3ae46f67e2224efd855e88e7cc33077d (127.29.62.253:41179), c90f19bd04294c5da80a6fc5328f428e (127.29.62.252:37601)
I20260808 17:50:06.368507 32655 raft_consensus.cc:348] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } }
I20260808 17:50:06.368677 32655 raft_consensus.cc:729] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90f19bd04294c5da80a6fc5328f428e, State: Initialized, Role: FOLLOWER
I20260808 17:50:06.368779 32655 consensus_queue.cc:260] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } }
I20260808 17:50:06.369109 32655 sys_catalog.cc:564] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:50:06.369138 32617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1ca74876bd69418bb1e253dce37a9092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c90f19bd04294c5da80a6fc5328f428e"
I20260808 17:50:06.369248 32617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:50:06.370190 32617 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca74876bd69418bb1e253dce37a9092 in term 1.
I20260808 17:50:06.370599 32344 leader_election.cc:302] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca74876bd69418bb1e253dce37a9092, c90f19bd04294c5da80a6fc5328f428e; no voters:
I20260808 17:50:06.370718 32669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:50:06.370877 32669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [term 1 LEADER]: Becoming Leader. State: Replica: 1ca74876bd69418bb1e253dce37a9092, State: Running, Role: LEADER
I20260808 17:50:06.370640 32702 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.368633 32320 master_runner.cc:186] Error getting master registration for 127.29.62.253:41179: OK, instance_id { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" instance_seqno: 1786211406148737 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:50:06.370936 32702 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:06.370976 32320 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:50:06.370962 32669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } }
I20260808 17:50:06.371443 32669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca74876bd69418bb1e253dce37a9092. Latest consensus state: current_term: 1 leader_uuid: "1ca74876bd69418bb1e253dce37a9092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.371553 32669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: This master's current role is: LEADER
I20260808 17:50:06.371766 32721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:50:06.372323 32721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:50:06.374212 32617 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [term 1 FOLLOWER]: Refusing update from remote peer 1ca74876bd69418bb1e253dce37a9092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:50:06.374534 32669 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:50:06.373528 32553 master_runner.cc:186] Error getting master registration for 127.29.62.253:41179: OK, instance_id { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" instance_seqno: 1786211406148737 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:50:06.376456 32553 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:50:06.376947 32702 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca74876bd69418bb1e253dce37a9092. Latest consensus state: current_term: 1 leader_uuid: "1ca74876bd69418bb1e253dce37a9092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.377059 32702 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:06.377861 32669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ca74876bd69418bb1e253dce37a9092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.377954 32669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: This master's current role is: LEADER
I20260808 17:50:06.378321 32702 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ca74876bd69418bb1e253dce37a9092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.378413 32721 catalog_manager.cc:1347] Generated new cluster ID: aca9a7da11eb42a7bebe0aa01127f9ad
I20260808 17:50:06.378461 32721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:50:06.378458 32702 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:06.378548 32722 mvcc.cc:205] Tried to move back new op lower bound from 7316321920506507264 to 7316321920496295936. Current Snapshot: MvccSnapshot[applied={T|T < 7316321920506507264 or (T in {7316321920506507264})}]
I20260808 17:50:06.378669 32723 mvcc.cc:205] Tried to move back new op lower bound from 7316321920506507264 to 7316321920496295936. Current Snapshot: MvccSnapshot[applied={T|T < 7316321920506507264}]
I20260808 17:50:06.382786 32542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } attempt: 3
W20260808 17:50:06.384424 32717 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:50:06.384490 32717 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 17:50:06 dist-test-slave-tk9z krb5kdc[32299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786211406, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.62.253@KRBTEST.COM for kudu/127.29.62.252@KRBTEST.COM
I20260808 17:50:06.391342 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.379137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.29.62.252:37601)
0808 17:50:06.379341 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.379343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.379352 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:50:06.379373 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.379374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.379383 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.379435 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.380443 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.381199 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.382716 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.383016 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.383053 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.387397 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.387398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.387398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.387406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.389488 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.389982 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.389983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.389984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.390062 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.390338 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.390339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.390339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.390543 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.390638 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.391132 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.391241 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260808 17:50:06.393292 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.383053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.62.252:37601 (local address 127.0.0.1:44192)
0808 17:50:06.383135 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 17:50:06.383145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:50:06.383180 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:50:06.383425 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:50:06.383426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:50:06.383617 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:50:06.383664 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.383666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.384609 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.384610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.385426 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:50:06.385429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.386005 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.386007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:50:06.386077 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.387714 (+ 1637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:50:06.387719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:50:06.389349 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:50:06.392058 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.392060 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.392060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.392302 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.392555 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:50:06.392556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:50:06.392556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:50:06.392625 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:50:06.392937 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:50:06.392938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:50:06.393000 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 17:50:06.393068 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 17:50:06.393183 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 17:50:06.393844 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.383248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.29.62.252:37601)
0808 17:50:06.383291 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.383293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.383300 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:50:06.383318 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.383319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.383328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.383383 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.383751 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.384509 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.385537 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.385871 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.385937 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.389456 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.389457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.389458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.389465 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.391363 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.392396 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.392397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.392397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.392478 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.392697 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.392698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.392699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.392845 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.392909 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.393631 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.393726 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:50:06.395632 32542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d: Bootstrap starting.
I20260808 17:50:06.396508 32542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d: Neither blocks nor log segments found. Creating new log.
I20260808 17:50:06.396904 32542 log.cc:824] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:50:06.400668 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.379184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.29.62.254:34075)
0808 17:50:06.379488 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.379490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.379498 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:50:06.379519 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.379520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.379530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.379580 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.380715 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.381452 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.391698 (+ 10246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.392023 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.392068 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.396362 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.396364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.396364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.396373 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.398434 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.399029 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.399031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.399031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.399095 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.399316 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.399317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.399317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.399476 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.399576 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.400066 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.400533 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260808 17:50:06.401532 32542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d: No bootstrap required, opened a new log
I20260808 17:50:06.402878 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:06.378880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33868 (local address 127.29.62.253:41179)
0808 17:50:06.378922 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.378924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.379489 (+ 565us) server_negotiation.cc:407] Connection header received
0808 17:50:06.392602 (+ 13113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.392604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.392619 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.392653 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.393753 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.394524 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.396060 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:06.396390 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:06.396433 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:06.399400 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:50:06.399401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:50:06.399402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:50:06.399410 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:50:06.401054 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.401738 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.401739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.401739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.401811 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:50:06.402139 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:50:06.402140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:50:06.402140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:50:06.402265 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:50:06.402332 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:06.402701 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 17:50:06.402785 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:50:06.403815 32542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } }
I20260808 17:50:06.403996 32542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae46f67e2224efd855e88e7cc33077d, State: Initialized, Role: FOLLOWER
I20260808 17:50:06.404098 32542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } }
I20260808 17:50:06.404469 32542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:50:06.404582 32504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:50:06.405571 32504 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [term 1 FOLLOWER]: Refusing update from remote peer 1ca74876bd69418bb1e253dce37a9092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:50:06.405985 32718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:50:06.406200 32505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1ca74876bd69418bb1e253dce37a9092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3ae46f67e2224efd855e88e7cc33077d"
I20260808 17:50:06.407053 32505 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca74876bd69418bb1e253dce37a9092 in term 1.
I20260808 17:50:06.409538 300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca74876bd69418bb1e253dce37a9092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.409633 300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:06.409718 300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca74876bd69418bb1e253dce37a9092. Latest consensus state: current_term: 1 leader_uuid: "1ca74876bd69418bb1e253dce37a9092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.409783 300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:06.409821 300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ca74876bd69418bb1e253dce37a9092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.409866 300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:50:06.411414 32766 mvcc.cc:205] Tried to move back new op lower bound from 7316321920506507264 to 7316321920496295936. Current Snapshot: MvccSnapshot[applied={T|T < 7316321920506507264}]
W20260808 17:50:06.412103 32764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:50:06.412163 32764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:50:06.418318 32718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ca74876bd69418bb1e253dce37a9092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca74876bd69418bb1e253dce37a9092" member_type: VOTER last_known_addr { host: "127.29.62.254" port: 34075 } } peers { permanent_uuid: "3ae46f67e2224efd855e88e7cc33077d" member_type: VOTER last_known_addr { host: "127.29.62.253" port: 41179 } } peers { permanent_uuid: "c90f19bd04294c5da80a6fc5328f428e" member_type: VOTER last_known_addr { host: "127.29.62.252" port: 37601 } } }
I20260808 17:50:06.418449 32718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092 [sys.catalog]: This master's current role is: LEADER
I20260808 17:50:06.419756 32721 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:50:06.420499 32721 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:50:06.434468 32721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ca74876bd69418bb1e253dce37a9092: Generated new TSK 0
I20260808 17:50:06.434710 32721 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:50:06.463797 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.462056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33870 (local address 127.29.62.253:41179)
0808 17:50:06.462105 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.462107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.463102 (+ 995us) server_negotiation.cc:407] Connection header received
0808 17:50:06.463215 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.463216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.463228 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.463259 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.463724 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33870: BlockingRecv error: recv got EOF from 127.0.0.1:33870 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 17:50:06.464740 32431 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.462511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33244 (local address 127.29.62.254:34075)
0808 17:50:06.462554 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.462556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.464221 (+ 1665us) server_negotiation.cc:407] Connection header received
0808 17:50:06.464321 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.464322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.464333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.464361 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.464674 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33244: BlockingRecv error: recv got EOF from 127.0.0.1:33244 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 17:50:06.465623 32662 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.462829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44194 (local address 127.29.62.252:37601)
0808 17:50:06.462871 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.462872 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.465070 (+ 2198us) server_negotiation.cc:407] Connection header received
0808 17:50:06.465211 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.465213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.465223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.465253 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.465563 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44194: BlockingRecv error: recv got EOF from 127.0.0.1:44194 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:50:06.475128 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.470454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33884 (local address 127.29.62.253:41179)
0808 17:50:06.470493 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.470495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.474543 (+ 4048us) server_negotiation.cc:407] Connection header received
0808 17:50:06.474659 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.474660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.474682 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.474716 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.475073 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33884: BlockingRecv error: recv got EOF from 127.0.0.1:33884 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:50:06.478300 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.477528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33894 (local address 127.29.62.253:41179)
0808 17:50:06.477563 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.477564 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.477808 (+ 244us) server_negotiation.cc:407] Connection header received
0808 17:50:06.477900 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.477901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.477910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.477939 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.478256 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33894: BlockingRecv error: recv got EOF from 127.0.0.1:33894 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:50:06.482164 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.481443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.29.62.253:41179)
0808 17:50:06.481474 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.481476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.481677 (+ 201us) server_negotiation.cc:407] Connection header received
0808 17:50:06.481767 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.481768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.481778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.481807 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.482124 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33898: BlockingRecv error: recv got EOF from 127.0.0.1:33898 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:50:06.488147 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.487346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.29.62.253:41179)
0808 17:50:06.487380 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.487382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.487611 (+ 229us) server_negotiation.cc:407] Connection header received
0808 17:50:06.487708 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.487709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.487719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.487749 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.488103 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33912: BlockingRecv error: recv got EOF from 127.0.0.1:33912 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:50:06.498502 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.497522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.29.62.253:41179)
0808 17:50:06.497562 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.497564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.497881 (+ 317us) server_negotiation.cc:407] Connection header received
0808 17:50:06.498009 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.498010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.498021 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.498052 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.498450 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33922: BlockingRecv error: recv got EOF from 127.0.0.1:33922 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:50:06.517016 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.516009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.29.62.253:41179)
0808 17:50:06.516056 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.516058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.516373 (+ 315us) server_negotiation.cc:407] Connection header received
0808 17:50:06.516486 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.516487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.516499 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.516530 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.516941 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33932: BlockingRecv error: recv got EOF from 127.0.0.1:33932 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 17:50:06.551461 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.550551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.29.62.253:41179)
0808 17:50:06.550596 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.550598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.550877 (+ 279us) server_negotiation.cc:407] Connection header received
0808 17:50:06.550989 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.550990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.551002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.551032 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.551398 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33938: BlockingRecv error: recv got EOF from 127.0.0.1:33938 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 17:50:06.617692 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.616960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.29.62.253:41179)
0808 17:50:06.617053 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.617059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.617140 (+ 81us) server_negotiation.cc:407] Connection header received
0808 17:50:06.617199 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.617200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.617210 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.617264 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.617630 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33940: BlockingRecv error: recv got EOF from 127.0.0.1:33940 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 17:50:06.747707 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:06.746872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.29.62.253:41179)
0808 17:50:06.746956 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:06.746958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:06.747086 (+ 128us) server_negotiation.cc:407] Connection header received
0808 17:50:06.747169 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:06.747170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:06.747189 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:06.747256 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:06.747625 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33950: BlockingRecv error: recv got EOF from 127.0.0.1:33950 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 17:50:07.005765 32549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:50:07.004998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.29.62.253:41179)
0808 17:50:07.005069 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:07.005072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:07.005123 (+ 51us) server_negotiation.cc:407] Connection header received
0808 17:50:07.005218 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:07.005223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:07.005233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:07.005298 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:50:07.005689 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33962: BlockingRecv error: recv got EOF from 127.0.0.1:33962 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 17:50:07.385012 32717 catalog_manager.cc:1259] Loaded cluster ID: aca9a7da11eb42a7bebe0aa01127f9ad
I20260808 17:50:07.385102 32717 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e: loading cluster ID for follower catalog manager: success
I20260808 17:50:07.385977 32717 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e: acquiring CA information for follower catalog manager: success
I20260808 17:50:07.386720 32717 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c90f19bd04294c5da80a6fc5328f428e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:50:07.412626 32764 catalog_manager.cc:1259] Loaded cluster ID: aca9a7da11eb42a7bebe0aa01127f9ad
I20260808 17:50:07.412720 32764 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d: loading cluster ID for follower catalog manager: success
I20260808 17:50:07.413594 32764 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d: acquiring CA information for follower catalog manager: success
I20260808 17:50:07.413939 32764 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ae46f67e2224efd855e88e7cc33077d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:50:07.521750 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:07.518939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.29.62.253:41179)
0808 17:50:07.519069 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:07.519072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:07.519141 (+ 69us) server_negotiation.cc:407] Connection header received
0808 17:50:07.519241 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:07.519243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:07.519310 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:07.519406 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:07.519577 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.520084 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.520876 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.521084 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.521159 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:07.521377 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:07.521492 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:07.521584 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:07.521622 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 17:50:07.521646 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260808 17:50:07.525789 414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:07.523547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.29.62.252:37601)
0808 17:50:07.523657 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:07.523659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:07.523667 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:50:07.523728 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:07.523729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:07.523763 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:07.523835 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:07.523939 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.524428 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.525103 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.525289 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.525338 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:07.525465 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:07.525562 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:07.525616 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:07.525653 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 17:50:07.525669 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":50,"threads_started":1}
I20260808 17:50:07.530407 414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:07.527627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44208 (local address 127.29.62.252:37601)
0808 17:50:07.527684 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:07.527686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:07.527823 (+ 137us) server_negotiation.cc:407] Connection header received
0808 17:50:07.527854 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:07.527855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:07.527866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:07.527913 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:07.528048 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.528526 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.529616 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.529894 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.529933 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:07.530127 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:07.530157 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:07.530212 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:07.530295 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 17:50:07.530321 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:50:07.530543 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:07.527430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.29.62.253:41179)
0808 17:50:07.527478 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:07.527481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:07.527667 (+ 186us) server_negotiation.cc:407] Connection header received
0808 17:50:07.527757 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:07.527758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:07.527770 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:07.527830 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:07.527984 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.528693 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.529657 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.529941 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.529996 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:07.530144 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:07.530179 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:07.530245 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:07.530424 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 17:50:07.530450 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:50:07.530710 426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:50:07.527450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33248 (local address 127.29.62.254:34075)
0808 17:50:07.527571 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 17:50:07.527574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:50:07.527686 (+ 112us) server_negotiation.cc:407] Connection header received
0808 17:50:07.527802 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:50:07.527803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:50:07.527856 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:50:07.527933 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:50:07.528115 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.528869 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.529605 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:50:07.529820 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:50:07.529895 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:50:07.530077 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:50:07.530174 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:50:07.530232 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:50:07.530436 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 17:50:07.530545 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260808 17:50:07.532582 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 32320
I20260808 17:50:07.541302 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 32440
I20260808 17:50:07.557257 29947 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGk3gfK/build/release/bin/kudu with pid 32553
2026-08-08T17:50:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3085 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6186 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (281937 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST